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CHAPTER  1 


STATEMENT  OF  THE  PROBLEM 
Introduction 

A  student  pilot  departs  on  a  solo  cross-country  flight.  The  primary  gyro 
compass  fails  half  way  through  the  flight  leaving  the  pilot  with  only  the  standby 
magnetic  compass  for  directional  reference.  The  novice  pilot  remembers  to  correct 
the  readings  for  the  deviation  of  the  instrument,  but  does  not  correct  for  difference 
between  the  earth’s  magnetic  and  north  poles  and  quickly  becomes  seriously 
disoriented.  Ultimately,  disorientation  and  low  fuel  produce  catastrophic 
consequences. 

In  another  incident,  a  motorist  in  a  borrowed  car  on  an  icy  road  begins  to 
skid.  Almost  instinctively,  the  driver  steers  into  the  skid  in  an  attempt  to  correct  the 
condition  as  he  would  have  done  in  his  own  car.  Unfortunately,  the  driver  owns  a 
rear  wheel  drive  vehicle,  and  the  borrowed  vehicle  is  a  front  wheel  drive.  The  skid 
becomes  aggravated,  and  the  car  slides  across  the  median  totally  out  of  control. 

What  do  these  two  incidents  have  in  common?  Both  cases  are  instances  of 
fragile  knowledge.  In  the  first  case,  the  novice  pilot  is  missing  critical  knowledge.  In 
the  second  case,  the  driver  inappropriately  applied  knowledge  from  one  situation  to 
an  entirely  different  situation.  What  is  fragile  knowledge? 
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Definition  of  Fragile  Knowledge 


According  to  David  Perkins  (1992,  p.  20)  fragile  knowledge  is  knowledge  that  a 
person  “does  not  remember,  understand,  or  actively  use  much  of  what  they  have 
supposedly  learned.”  This  type  of  knowledge  has  little  substance  or  strength  and  is 
flimsy  or  weak  when  demands  are  made  on  it.  Research  shows  that  fragile 
knowledge  can  manifest  itself  in  several  forms.  The  six  most  common  types  of 
fragile  knowledge  are: 

•  inert  knowledge  (Whitehead,  1929) 

•  partial  or  missing  knowledge  (Perkins  &  Martin,  1985,  p.  5) 

•  misplaced  knowledge  (Perkins  &  Martin,  1985,  p.  5) 

•  naive  knowledge  (Perkins  &  Martin,  1985,  p.  5) 

•  conglomerative  knowledge  (Perkins  &  Martin,  1985,  p.  5) 

•  ritual  knowledge  (Taba  &  Elzey,  1964,  p.  524) 

Perkins  (1992)  claims  that  these  six  types  of  fragile  knowledge  can  combine  in  a 

learner  causing  the  display  of  a  distinctive  cluster  of  behaviors.  This  distinctive 

cluster  of  behaviors  has  been  labeled  fragile  knowledge  syndrome.  The  following 

example  from  Perkins  (1992)  illustrates  this  syndrome. 

Brian  is  tackling  fraction  computation  problems.  For  the  simplest 
problems,  Brian  proceeds  nicely.  Encountering  a  mixed  number, 

Brian  has  no  idea  what  to  do  with  it— a  knowledge  gap.  On  another 
problem,  Brian  obtains  an  answer  that  needs  reducing  but  forgets  to 
reduce  it,  even  though  he  knows  how.  On  an  addition  problem,  Brian 
cancels  a  3  in  the  numerator  of  one  term  against  a  3  in  the 
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denominator  of  the  other,  mistakenly  believing  that  cancellation 
works  for  sums  as  well  as  products.  However,  on  a  similar  problem, 
he  does  not  try  canceling  and  solves  the  problem  correctly,  (p.  26) 

The  performance  in  this  example  is  inconsistent.  At  times,  the  learner  seems 

competent.  However,  at  other  times,  with  similar  problems  the  learner  reaches  an 

impasse  or  elects  to  perform  an  inappropriate  action.  The  cause  of  the  impasses  or 

inappropriate  actions  can  be  traced  to  one  of  several  common  types  of  fragile 

knowledge. 

To  provide  a  little  more  detail  on  the  types  of  fragile  knowledge,  “inert 
knowledge  refers  to  situations  where  the  student  fails  to  retrieve  knowledge  but  in 
fact  possesses  it,  as  revealed  by  a  clinical  probe”  (Perkins  &  Martin,  1985,  p.  5). 
Partial  knowledge  “is  the  straightforward  case  of  an  impasse  due  to  knowledge  the 
student  has  not  retained  or  ever  learned,  as  revealed  by  clinical  probes  failing  to 
reveal  signs  of  the  knowledge”  (Perkins  &  Martin,  1985,  p.  5).  Misplaced 
knowledge  is  an  instance  of  a  student  importing  knowledge  appropriate  to  one 
context  but  inappropriate  to  the  new  context.  Naive  knowledge  is  knowledge  that 
takes  the  form  of  naive  theories  and  stereotypes  (Perkins,  1985,  p.  5).  A  “flat  earth” 
is  a  common  example  of  this  type  of  knowledge.  Conglomerative  knowledge 
signifies  an  attempt  by  the  student  to  force  together  disparate  elements  of  knowledge 
to  provide  a  solution  to  a  problem,  repair  a  bug,  or  resolve  an  impasse.  Finally,  ritual 
knowledge  is  knowledge  that  is  culturally  defined  and  usually  limited  to  a  particular 
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context;  for  example,  knowledge  or  tasks  that  are  useful  in  school  but  not  anywhere 
else. 

Educators  have  an  ethical  responsibility  to  minimize  the  construction  of 
fragile  knowledge.  Are  there  things  that  we  can  do  as  educational  professionals  to 
minimize  or  inhibit  the  learners’  construction  of  fragile  knowledge?  Does  how  we 
interact  with  learners  in  an  instructional  context  influence  what  they  learn?  Do 
instructional  interaction  strategies  that  we  use  with  students  influence  what  they 
learn?  If  the  answer  is  yes,  then  are  some  strategies  more  effective  then  others  in 
minimizing  or  inhibiting  particular  types  of  fragile  knowledge  construction?  We 
have  a  responsibility  to  our  clients  and  students  to  seek  answers  to  these  questions. 

Purpose  of  the  Study 

This  study  examined  the  effects  of  four  common  instructional  interaction 
strategies  and  their  effects  on  the  learners’  construction  of  fragile  knowledge  during 
initial  procedural  skill  acquisition.  The  study  also  examined  the  effect  of  providing 
explicit  instruction  on  executive  control  processes  and  strategies  during  initial 
procedural  skill  acquisition.  Executive  control  processes  are  those  processes 
involved  in  planning,  monitoring,  and  evaluating  performance  on  intellectual  tasks 
(Vygotsky,  1978). 

This  research  effort  addressed  the  following  questions:  Does  the  way  we 
interact  with  learners  have  an  effect  on  the  learners’  construction  of  fragile 
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knowledge?  If  so,  are  some  interaction  strategies  more  effective  than  others  at 
minimizing  or  inhibiting  the  construction  of  particular  types  of  fragile  knowledge? 

Also,  previous  research  has  suggested  that  direct  instruction  on  executive 
control  processes  may  reduce  fragile  knowledge  construction  in  problem-based 
learning  environments  (Perkins  &  Martin,  1985).  Can  direct  instruction  on  executive 
control  processes  reduce  fragile  knowledge  construction  during  acquisition  of 
algorithmic-heuristic  cognitive  skills? 

Finally,  is  there  an  interaction  between  the  use  of  common  instructional 
interaction  strategies  and  the  explicit  instruction  of  executive  control  processes?  Are 
there  certain  combinations  of  interaction  strategies  and  executive  control  processes 
that  together  produce  a  significant  difference  in  the  amount  of  fragile  knowledge 
construction? 

Experimental  Intervention 

The  design  of  this  experiment  was  a  4  X  2  factorial  design.  The  two 
independent  variables  were  instructional  interaction  strategies  (with  four  levels)  and 
executive  control  strategies  (with  two  levels). 

The  combination  of  levels  of  the  two  independent  variables  resulted  in  eight 
treatments  being  administered  in  this  experiment.  The  eight  treatments  were: 

1.  No  instructional  interaction  with  executive  controls:  Subjects  in  this 
treatment  group  were  given  a  coursebook  which  contained  the  information  necessary 
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to  complete  the  specific  procedures  addressed  by  each  particular  learning  objective. 
After  receiving  a  coursebook,  the  subjects  were  sent  to  the  simulator  to  complete  the 
simulation  without  any  instructional  assistance.  Their  coursebook  contained  explicit 
instruction  on  executive  control  strategies  applicable  to  each  procedure  they  were 
required  to  perform. 

2.  No  instructional  interaction  without  executive  controls:  These  subjects 
were  given  a  coursebook  and  sent  to  the  simulator  to  complete  the  lesson  without 
any  instructional  assistance.  Their  coursebook  did  not  contain  any  instruction  on 
executive  control  strategies. 

3.  Directive/didactic  instructional  interaction  with  executive  controls: 
Instructors  in  this  group  directed  subjects  through  each  procedure.  Each  subject  was 
told  in  a  direct  step-by-step  manner  how  to  accomplish  each  of  the  required  tasks. 
The  instructional  interaction  strategy  focused  on  “telling”  or  making  directive  or 
declarative  statements  to  the  learner  to  perform  each  step  of  the  procedure. 
Instructors  in  this  group  also  provided  explicit  (but  still  directive)  instruction  on 
executive  control  of  the  processes  within  the  procedures. 

4.  Directive/didactic  instructional  interaction  without  executive  controls: 
Subjects  in  this  group  received  the  same  type  of  interaction  as  the  previous  group, 
but  they  did  not  receive  any  instruction  on  executive  control  of  the  processes  within 
the  procedures. 
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5.  Questioning/Reflective  instructional  interaction  with  executive  controls; 
Instructors  in  this  group  used  questions  to  guide  students  through  the  same  tasks  as 
the  other  groups.  They  also  used  questions  to  lead  the  student  through  executive 
control  strategies  that  could  be  used  to  resolve  impasses  or  check  the  output  of  the 
procedures  being  performed. 

6.  Questioning/Reflective  instructional  interaction  without  executive 
controls:  This  group  experienced  the  same  interaction  style  as  the  previous  group, 
but  it  did  not  receive  the  additional  questioning  on  executive  control  strategies. 

7.  Mixed  Directive/Reflective  instructional  interaction  with  executive 
controls:  Instructors  in  this  group  presented  the  procedures  in  a  directive  style  first, 
and  then  they  shifted  to  a  questioning  strategy  as  the  subject  performed  the  skill. 
Subjects  in  the  other  instructor-assisted  groups  were  simply  directed  through  the 
procedure  a  second  time  to  ensure  equal  time-on-task  for  all  treatment  groups. 
Subjects  in  this  group  also  received  explicit  instruction  (in  a  questioning  type 
interaction  style)  on  executive  control  strategies. 

8.  Mixed  Directive/Reflective  instructional  interaction  without  executive 
controls:  Subjects  in  this  group  received  the  same  style  of  interaction  as  the  previous 
group,  but  they  didn’t  receive  any  instruction  on  executive  control  strategies. 

All  eight  treatments  were  conducted  within  the  context  of  two  actual 
navigation  simulation  missions.  These  missions  were  part  of  a  current  course  on 
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navigation  taught  at  the  United  States  Air  Force  Academy.  The  subjects  were  taking 
their  first  air  navigation  course.  Each  subject  received:  (1)  a  pretest,  (2)  two 
navigation  simulation  missions  with  instruction  according  to  their  treatment  group, 
(3)  and  an  immediate  posttest.  The  posttest  measured  how  much  fragile  knowledge 
on  basic  air  navigation  principles  and  procedures  the  subject  still  possessed  after  the 
instructional  treatment.  The  number  of  instances  of  fragile  knowledge  of  basic  air 
navigation  principles  and  procedures  was  the  dependent  variable  in  this  study. 

Research  Hypotheses 

The  primary  hypothesis  stated  that  subjects  who  were  provided  instructional 
interaction  would  perform  better  on  the  posttest  than  subjects  who  were  not  provided 
instructional  interaction.  Better  performance  (higher  scores)  on  the  posttest  indicated 
less  fragile  knowledge  construction.  The  reason  for  this  is  that  expert  guidance 
during  the  knowledge  compilation  process  in  the  associative  stage  of  cognitive  skill 
development  should  have  an  effect  on  what  the  learner  encodes.  Including  input 
from  experts  should  clarify  what  actually  gets  encoded  during  the  compilation 
process,  thus  strengthening  the  semantic  connections  the  learner  makes. 
Strengthening  the  semantic  connections  should  reduce  the  likelihood  of  constructing 
a  tenuous  grasp  on  the  knowledge. 

A  secondary  hypothesis  stated  that  subjects  who  were  provided  explicit 
instruction  on  executive  control  processes  would  score  better  on  the  posttest  than 
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subjects  who  weren’t  provided  this  instruction.  Once  again,  a  better  score  on  the 
posttest  indicated  a  lower  amount  of  fragile  knowledge  present  in  the  learner.  The 
reason  for  this  hypothesis  is  that  the  possession  and  use  of  executive  control 
strategies  should  enable  learners  to  solve  their  own  impasses  and  gain  a  stronger 
grasp  on  the  knowledge  to  be  gained.  Subjects  who  were  unable  to  resolve  their  own 
impasses  would  not  even  have  access  to  the  additional  knowledge  opportunities  that 
existed  beyond  the  initial  point  of  their  impasse. 

Significance  of  the  Study 

This  study  provided  evidence  for  educational  professionals  concerning  the 
issue  of  the  influence  of  common  instructional  interaction  strategies  on  their  ability 
to  reduce,  minimize,  or  inhibit  the  learners’  construction  of  fragile  knowledge  during 
initial  procedural  skill  acquisition.  Knowing  whether  these  strategies  have  an  effect 
on  knowledge  construction  or  which  strategies  are  more  effective  than  others  in 
reducing  the  construction  of  fragile  knowledge  will  enable  educational  professionals 
to  make  improvements  to  products  and  processes  that  will  reduce  the  construction  of 
fragile  knowledge  and  its  consequences. 

Definition  of  Terms 

Cognitive  Strategies:  “skills  by  which  learners  regulate  their  own  internal  processes 
of  attending,  learning,  remembering,  and  thinking”  (Gagne,  1985,  p.  55). 
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Conglomerative  Knowledge:  signifies  an  attempt  by  the  student  to  force  together 
disparate  elements  of  knowledge  to  provide  a  solution  to  a  problem,  repair  a  bug,  or 
resolve  an  impasse  (Perkins  and  Martin,  1985,  p.  7). 

Executive  Control  Processes;  are  “those  processes  involved  in  planning 
monitoring,  and  evaluating  performance  on  intellectual  tasks”  (Sincoff  &  Sternberg, 
1989,  p.  33). 

Executive  Control  Strategies:  are  a  superset  of  cognitive  strategies.  According  to 
Robert  M.  Gagne  (1985,  p.  77)  “they  are  a  part  of  the  information-processing  theory 
of  learning  and  memory  that  influences  attention  and  selective  perception  which 
determine  what  is  encoded  into  short  and  ultimately  long-term  memory.”  The 
executive  control  strategies  used  in  this  study  included  additional  procedural  tools 
that  were  used  to  control  and  cross-check  the  execution  of  the  navigation  procedure. 
These  strategies  included:  close  tracking  of  the  procedure  being  executed,  strategic 
orientation,  and  self-prompting. 

Fragile  Knowledge;  David  Perkins  (1985,  p.  5)  defines  fragile  knowledge  as 
knowledge  that  “students  do  not  remember,  understand,  or  use  actively  much  of 
what  they  have  supposedly  learned;  and  poor  thinking,  which  means  that  students  do 
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not  think  very  well  with  what  they  know.”  This  type  of  knowledge  has  little 
substance  or  strength  and  is  flimsy  or  weak  when  demands  are  put  on  it. 

Inert  Knowledge:  refers  to  “situations  where  the  student  fails  to  retrieve  knowledge 
but  in  fact  possesses  it,  as  revealed  by  a  clinical  probe”  (Perkins,  1985,  p.  5). 

Instructional  Interaction  Strategy:  a  superset  of  elemental  methods  or  tactics  for 
conveying  instruction  to  the  learner  and/or  for  receiving  and  responding  to  input 
from  the  learner.  Instructional  interaction  strategies  are  similar  to  instructional 
delivery  strategies,  but  they  focus  exclusively  on  the  instructor/leamer  exchange. 
Some  typical  strategies  include  directive  or  didactic  strategies,  questioning  or 
reflective  strategies,  or  combinations  of  the  two. 

Misplaced  Knowledge:  is  an  instance  of  a  student  importing  knowledge 
appropriate  to  another  context  but  inappropriate  to  the  context  it  is  being  imported 
to. 

Naive  Knowledge:  is  knowledge  that  takes  the  form  of  naive  theories  and 
stereotypes  (Perkins,  1985,  p.  5).  A  “flat  earth”  is  a  common  example  of  this  type  of 
knowledge. 
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Partial  Knowledge:  “is  the  straightforward  case  of  an  impasse  due  to  knowledge  the 
student  has  not  retained  or  ever  learned,  as  revealed  by  clinical  probes  failing  to 
reveal  signs  of  the  knowledge”  (Perkins,  1985,  p.  5). 

Ritual  Knowledge;  is  knowledge  that  is  culturally  defined  and  usually  limited  to  a 
particular  context;  for  example,  knowledge  or  tasks  that  are  useful  in  school  but  not 
anywhere  else  (Taba  &  Elzey,  1964). 
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CHAPTER  2 


REVIEW  OF  LITERATURE 
Introduction 

The  concept  and  consequences  of  fragile  knowledge  are  not  new.  However, 
systematic  research  of  fragile  knowledge  is  limited  in  scope  and  dates  back  only  ten 
years.  What  insights  into  the  nature  of  fragile  knowledge  do  we  currently  possess? 
We  must  assess  the  current  state  of  our  knowledge  before  finding  and  developing 
new  techniques  to  minimize  construction  of  fragile  knowledge  and  reduce  our 
exposure  to  its  consequences. 

This  chapter  surveys  the  literature  on  fragile  knowledge  and  examines  the 
broader  domain  of  procedural  skill  acquisition.  It  starts  with  the  theoretical 
limitations  of  the  scope  of  the  survey  of  the  literature,  and  then  it  briefly  delineates 
the  philosophical  assumptions  of  the  study.  With  the  theoretical  foundation  of  the 
review  set,  it  begins  by  examining  the  definition  of  fragile  knowledge,  and  then  it 
seeks  to  define  cognitive  skill.  Once  the  concept  of  cognitive  skill  is  defined,  the 
focus  then  shifts  to  the  acquisition  of  cognitive  skill  and  to  the  origins  and 
development  of  procedural  skill.  The  theoretical  portion  of  the  review  concludes 
with  an  examination  of  executive  control  processes  and  how  they  can  effect  the 
procedural 
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skill  acquisition  process.  Finally,  literature  on  common  instructional  interaction 
strategies  concludes  the  chapter,  establishes  the  theoretical  rationale  for  the  study, 
and  provides  the  connection  to  previous  research  on  the  topic  of  fragile  knowledge 
and  its  effects  on  learners. 

Theoretical  Limitations 

Fragile  knowledge  is  a  broad  concept  with  an  infinite  number  of  possible 
contexts.  This  review  is  limited  to  literature  pertaining  to  fragile  knowledge 
construction  during  initial  algorithmic-heuristic  procedural  skill  acquisition  as 
defined  by  Colley  and  Beech  (1989).  Algorithmic-heuristic  procedural  skills  are  a  set 
sequence  of  steps  a  performer  uses  to  arrive  at  a  solution  (Colley  &  Beech,  1989). 
Only  the  cognitive  domain  is  addressed;  the  affective  and  psychomotor  domains  are 
not  reviewed.  Furthermore,  issues  of  transfer  and  retention  of  procedural  skills  are 
also  ignored.  This  is  not  to  imply  that  the  domains  or  issues  not  covered  are  not 
important;  they  are  simply  not  of  interest  to  this  particular  study.  This  study  focuses 
on  the  construction  of  fragile  knowledge  at  the  point  of  initial  cognitive  procedural 
skill  acquisition. 

Philosophical  Assumptions 

In  another  effort  to  stabilize  the  theoretical  foundation,  an  information 
processing  model  (Gagne,  1974,  1985)  was  assumed  throughout  this  study.  The 
information  processing  model  was  chosen  because  of  its  broad  representation  in  the 
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literature  on  procedural  skill  acquisition  (Anderson,  1980).  Furthermore,  information 
processing  models  address  the  process  of  skill  acquisition  which  is  the  focus  of  this 
study.  Although  the  information  processing  model  cannot  completely  explain  the 
complexities  and  intricacies  associated  with  more  holistic  views  on  learning,  the 
model  does  provide  a  practical  means  to  communicate  observations  of  a  complex 
phenomenon  such  as  cognitive  procedural  skill  acquisition  between  researchers  and 
consumers  of  research. 

Information-processing  theory  grew  out  of  human  factors  work  and 
information  theory;  it  gained  prominence  during  the  reemergence  of  cognitive 
psychology  after  World  War  II  (Anderson,  1980).  Newell  and  Simon’s  work  during 
the  1960s  and  1970s  on  artificial  intelligence  at  Canegie-Mellon  has  also  strongly 
influenced  the  information-processing  model. 

Information  processing  approaches  to  explaining  human  cognition  center 
around  the  analogy  between  how  a  computer  is  programmed  to  process  information 
and  how  a  human  processes  information.  Cognitive  psychologists  who  hold  this 
view  see  the  human  mind  as  a  computer  (Andre  &  Phye,  1986).  “Information¬ 
processing  models  suggest  that  people  ‘store’  concepts  and  rules  in  a  place  called 
memory  through  a  vast  network  or  schema  of  interrelated  symbols,  propositions,  and 
knowledge  components”  (Wilson  &  Teslow,  1992,  p.  1).  According  to  John 
Anderson  (1980,  p.  13), 
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there  is  a  clear  sequence  or  serial  ordering  to  mental  operations, 
and  the  important  characteristic  of  an  information-processing 
analysis,  then,  is  that  it  involves  a  tracing  of  a  sequence  of  mental 
operations  and  their  products  (information)  in  the  performance  of  a 
particular  cognitive  task 

By  this  statement,  Anderson  asserts  that  there  is  a  sequence  of  mental  operations  that 
convert  raw  sensory  information  into  knowledge.  Do  changes  in  the  mental 
operations  in  this  sequence  produce  corresponding  changes  in  the  information  that  is 
converted  into  knowledge?  If  so,  can  these  variations  in  information  processing 
produce  fragile  knowledge? 

Definition  of  Fragile  Knowledge 

The  first  and  most  obvious  question  is  what  is  fragile  knowledge?  Fragile 
knowledge  is  knowledge  that  “students  do  not  remember,  understand,  or  use  actively 
much  of  what  they  have  supposedly  learned;  and  poor  thinking,  which  means  that 
students  do  not  think  very  well  with  what  they  know”  (Perkins,  1992,  p.  20). 
Research  shows  that  fragile  knowledge  can  manifest  itself  in  several  forms; 
however,  these  various  forms  can  also  combine  in  a  learner  which  results  in  the 
display  of  a  distinctive  cluster  of  behaviors  called  fragile  knowledge  syndrome 
(Perkins,  1992,  p.  26).  Several  common  forms  of  fragile  knowledge  are; 

•  inert  knowledge  (Whitehead,  1929) 

•  partial  or  missing  knowledge  (Perkins  &  Martin,  1985,  p.  5) 

•  misplaced  knowledge  (Perkins  &  Martin,  1985,  p.  5) 
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•  naive  knowledge  (Perkins  &  Martin,  1985,  p.  5) 

•  conglomerative  knowledge  (Perkins  &  Martin,  1985,  p.  5) 

•  ritual  knowledge  (Taba  &  Elzey,  1964,  p.  524) 

To  provide  a  little  more  detail,  “inert  knowledge  refers  to  situations  where 
the  student  fails  to  retrieve  knowledge  but  in  fact  possesses  it,  as  revealed  by  a 
clinical  probe”  (Perkins,  1985,  p.  5).  In  aviation,  an  important  ability  that  every  pilot 
must  possess  is  the  ability  to  determine  when  the  aircraft  will  arrive  at  a  specific 
geographic  point.  Estimating  the  time  of  arrival  accurately  facilitates  smooth 
coordination  with  ground  support  personnel  and  enables  accurate  monitoring  of  fuel 
requirements  for  the  flight.  Inability  to  recall  the  first  step  of  how  to  compute  an 
estimated  time  of  arrival  (ETA)  during  the  simulation  is  an  example  of  inert 
knowledge  within  the  context  of  this  study. 

Partial  knowledge  “is  the  straightforward  case  of  an  impasse  due  to 
knowledge  the  student  has  not  retained  or  ever  learned,  as  revealed  by  clinical 
probes  failing  to  reveal  signs  of  the  knowledge”  (Perkins,  1985,  p.  5).  The  symptoms 
of  partial  knowledge  are  similar  to  the  symptoms  of  inert  knowledge.  The  key  to 
distinguishing  between  the  two  types  is  that  in  the  case  of  inert  knowledge  probing 
by  the  instructor  reveals  that  the  student  does  in  fact  remember  the  information.  In 
the  case  of  partial  knowledge,  probing  questions  used  by  the  instructor  fail  to  reveal 
that  the  student  actually  possessed  the  knowledge  in  the  first  place. 
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Misplaced  knowledge  is  an  instance  of  a  student  importing  knowledge 
appropriate  to  a  particular  context  but  inappropriate  to  the  new  context.  Attempting 
to  compute  a  groundspeed  for  the  ETA  computation  instead  of  reading  it  from  the 
Navigation  Computer  System  is  an  example  of  misplaced  knowledge.  Computing  a 
groundspeed  is  appropriate  for  flight  planning,  but  it  is  inappropriate  for  ETA 
computation. 

Naive  knowledge  is  knowledge  that  takes  the  form  of  naive  theories  and 
stereotypes  (Perkins,  1985).  A  “flat  earth”  is  a  common  example  of  this  type  of 
knowledge.  We  know  that  the  earth  is  not  flat;  it  is  round.  You  cannot  fly  off  the  end 
of  the  earth.  Using  true  airspeed  instead  of  groundspeed  for  the  ETA  computation  is 
an  indication  of  the  presence  of  naive  knowledge. 

Conglomerative  knowledge  signifies  an  attempt  by  the  student  to  force 
together  disparate  elements  of  knowledge  to  provide  a  solution  to  a  problem,  repair  a 
bug,  or  resolve  an  impasse.  Measuring  a  distance  from  a  dead  reckoning  position 
instead  of  measuring  the  distance  from  a  navigational  fix  (as  the  ETA  computation 
procedure  calls  for)  simply  to  be  able  to  come  up  with  a  possible  answer  for  the  ETA 
problem  is  indicative  of  conglomerative  knowledge. 

Finally,  ritual  knowledge  is  knowledge  that  is  culturally  defined  and  usually 
limited  to  a  particular  context;  for  example,  knowledge  or  tasks  that  are  useful  in 
school  but  not  anywhere  else.  Teaching  students  to  use  a  slide  rule  to  perform 
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mathematical  calculations  when  the  “real  world”  uses  electronic  calculators  is  an 
example  of  ritual  knowledge.  All  tasks  performed  in  the  study  were  real-world  tasks. 
None  of  the  concepts  or  procedures  were  for  training  purposes  only.  Therefore,  ritual 
knowledge  was  not  relevant  to  this  study. 

All  of  the  concepts  mentioned  here  are  manifestations  or  instances  of  fragile 
knowledge.  Fragile  knowledge,  as  mentioned  previously,  is  a  significant  deterrent  to 
the  successful  performance  of  any  cognitive  skill.  What  is  a  cognitive  skill?  How  is 
it  acquired?  Answers  to  these  questions  will  yield  an  insight  into  the  nature  of 
cognitive  skills  and  how  to  minimize  the  effects  of  fragile  knowledge  during  the 
initial  acquisition  of  a  cognitive  skill. 

Definition  of  Cognitive  Skill 

A  cognitive  skill  is  the  ability  to  perform  an  intellectual  procedure 
(Anderson,  1980).  Cognitive  skill  encompasses  two  categories  of  knowledge.  The 
first  category  of  knowledge  in  a  cognitive  skill  is  declarative  knowledge.  Declarative 
knowledge  is  “knowing  that.”  The  second  category  is  procedural  knowledge. 
Procedural  knowledge  is  “knowing  how  (Gagne,  1985,  p.  48).”  Possessing  both 
categories  of  knowledge  is  paramount  to  the  ability  to  perform  a  cognitive  skill. 

Now  that  we  know  what  a  cognitive  skill  is,  how  is  it  acquired? 


19 


Acquisition  of  Cognitive  Skill 


Recent  approaches  to  examining  skill  acquisition  focus  on  the  works  of  John 
Anderson  from  Carnegie-Mellon.  According  to  Colley  and  Beech  (1989),  Anderson 
provides  a  framework  for  understanding  observations  made  earlier  by  Fitts  (1964)  on 
the  development  of  skill.  Colley  and  Beech  (1989)  outline  Fitts’  three  stages  of 
cognitive  skill  development: 

the  cognitive  stage,  in  which  the  learner  makes  an  initial  approximation 
to  the  skill,  based  upon  background  knowledge,  observation  or  instruction; 
the  associative  stage,  in  which  performance  is  refined  through  the 
elimination  of  errors;  and  the  autonomous  stage,  in  which  skilled 
performance  is  well-established  but  still  continues  to  improve,  albeit 
very  gradually,  (p.  7) 

Anderson’s  (1982)  adaptive  control  of  thought  (ACT)  theory  posits  two 
stages  of  skill  development.  His  first  stage  is  the  declarative  stage.  In  this  stage, 
declarative  representations  relevant  to  the  skill  guide  behavior.  The  second  stage  is 
the  procedural  stage  where  knowledge  undergoes  a  process  of  conversion  and 
continual  refinement  of  conditions  and  actions  into  procedures.  He  calls  the  process 
of  converting  facts  into  procedures  knowledge  compilation  (Colley  and  Beech, 
1989).  Knowledge  compilation  also  results  in  an  increase  in  the  rate  of  production 
application  (condition-action  pairs  being  applied  to  solve  procedural  problems)  and 
tuning  processes.  This  becomes  the  basis  for  improving  procedural  skill  performance 
and  facilitating  the  transition  from  novice  to  expert. 
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In  Anderson’s  declarative  stage,  knowledge  about  how  to  perform  a  skill  is 
assembled  from  declarative  memory,  and  from  instruction  or  guidance,  into  working 
memory.  General  problem-solving  procedures  then  turn  this  declarative  knowledge 
into  productions  which  are  composed  of  condition-action  pairs.  Rummelhart  and 
Norman  (1981)  discuss  the  utility  of  analogy  to  create  or  extend  schemata’s  ability 
to  incorporate  task  knowledge  in  procedures.  How  is  declarative  knowledge 
converted  into  procedural  knowledge?  It  happens  through  a  process  called 
knowledge  compilation.  Knowledge  of  the  compilation  and  tuning  processes 
provides  a  conceptual  framework  for  understanding  how  and  why  instructional 
interaction  may  have  an  effect  on  knowledge  construction. 

Knowledge  compilation  occurs  between  the  declarative  and  procedural  stages 
of  cognitive  skill  acquisition.  Knowledge  composition  has  two  subprocesses; 
composition  and  proceduralization.  Composition  (Lewis,  1978)  collapses  successive 
productions  (condition-action  pairs)  into  a  single  production  which  has  the  same 
effect.  Proceduralization  removes  clauses  in  the  condition  of  a  production  that 
require  matching  from  long-term  memory  via  working  memory.  This  removal  of 
clauses  permits  automatization  of  the  procedural  task.  Compilation  is  a  gradual 
process  which  allows  for  errors  in  procedural  information  to  be  corrected  over 
practice. 


In  the  procedural  stage  of  acquisition,  productions  are  tuned,  that  is,  made 
more  appropriate  and  efficient  for  the  task  at  hand.  Tuning  has  three  subprocesses: 
generalization,  discrimination,  and  strengthening.  Generalization  occurs  when 
common  aspects  of  specific  productions  are  used  to  create  a  more  widely  applicable 
production  which  can  be  used  in  novel  situations.  Discrimination  restricts  the  use  of 
productions  to  instances  where  they  are  successful.  Strengthening  involves  repeating 
applications  so  that  the  time  taken  to  apply  them  diminishes  (Colley  &  Beech,  1989). 
Now  that  we  understand  the  cognitive  skill  acquisition  process,  it’s  time  to  look  at 
the  procedural  skill  acquisition  process. 

Origins  and  Development  of  Procedural  Skill 
With  the  broader  context  of  cognitive  skills  established,  the  focus  now 
narrows  to  procedural  skills  which  are  a  subset  of  cognitive  skill.  Although  some 
controversy  and  a  few  alternative  theories  exist,  Anderson’s  (1982)  adaptive  control 
of  thought  (ACT)  theory  remains  the  seminal  work  on  procedural  skill  acquisition. 
The  controversy  centers  around  Anderson’s  claim  that  knowledge  must  be  encoded 
declaratively  before  proceduralization  can  occur  and  that  proceduralization  can  only 
occur  as  a  by  product  of  actually  performing  the  procedure.  Kieras  and  Bovair 
(1985)  disagree.  Their  study  claims  that  learners  can  represent  production  rules  (the 
If  (condition)  THEN  (action)  statements)  declaratively  before  they  are 
proceduralized  without  having  to  actually  perform  the  procedure.  In  spite  of  the 
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disagreement  on  when  a  production  can  be  proceduralized  there  is  agreement  on 
what  actually  happens  in  the  process. 

According  to  ACT  theory  (Anderson,  1982)  the  procedural  skill  acquisition 
process  involves  three  distinct  steps.  These  steps  are  encoding,  proceduralization, 
and  composition  (Neves  &  Anderson,  1981).  All  incoming  knowledge  is  encoded 
declaratively  as  a  set  of  facts  in  a  semantic  network.  This  information  is  then 
proceduralized  by  converting  it  into  productions  which  are  condition-action  pairs. 
These  productions  created  during  the  proceduralization  phase  are  then  subject  to  the 
composition  process.  During  the  composition  process  productions  are  combined  into 
larger  productions  to  decrease  the  amount  of  time  required  to  apply  a  procedure. 


Anderson  (1981,  p.  67)  provides  the  following  illustration  of  composition; 


Production  12:  If 
Then 


you  see  a  red  light 
assert  danger. 


Production  13:  If 
Then 


there  is  danger  and  another  person  is  near  you 
warn  that  person. 


The  simple  composite  is: 


Production  14:  If  you  see  a  red  light 
and  there  is  danger 
and  another  person  is  near  you 


Then  assert  danger 

and  warn  that  person 
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However,  this  process  is  not  without  its  problems.  Problems  that  result  in 

bugs  or  impasses  can  occur  at  any  point  during  the  acquisition  process.  Burton  and 

Brown  (1982)  conducted  extensive  research  on  this  subject  with  their  classic 

program  Debuggy.  Bugs  are  systematic  errors  in  the  execution  of  procedural  skills, 

and  these  errors  produce  fragile  knowledge.  Burton  and  Brown  (1982),  VanLehn 

(1982),  Shaw  (1982),  and  Sleeman  (1984)  observed  and  documented  over  200 

different  kinds  of  bugs  in  the  acquisition  of  simple  mathematical  skills  alone.  Bugs 

indicate  the  presence  of  fragile  knowledge  in  the  learner. 

VanLehn  (1982)  claims  that  mysteries  abound  in  bug  data  and  looks  to 

answers  to  the  following  questions  as  central  to  solving  these  mysteries. 

Why  are  there  so  many  different  bugs?  What  causes  them? 

What  causes  bugs  to  migrate  or  disappear?  Why  do  certain  bugs 
only  migrate  to  certain  other  bugs?  Often  a  student  has  more  than 
one  bug  at  a  time — why  do  certain  bugs  almost  always  occur  together? 

Do  co-occurring  bugs  have  the  same  cause?  Most  important,  how 
is  the  educational  process  involved  in  the  development  of  bugs?  (p.  49) 

VanLehn  appears  to  believe  that  a  correlation  may  exist  between  the  educational 

process  and  the  development  of  bugs  that  produce  fragile  knowledge.  Since  bugs 

indicate  the  presence  of  fragile  knowledge  in  the  learner,  can  the  educational  process 

eliminate  or  reduce  these  bugs?  Are  there  things  educators  can  do  to  facilitate  bug- 

free  procedural  skill  learning? 
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Executive  Control  Processes 


Much  of  VanLehn’s  work  focuses  on  impasses  caused  by  bugs  and  points  to 
executive  control  processes  to  solve  these  impasses  and  repair  the  bugs  that  caused 
them.  Executive  control  processes  facilitate  the  planning,  monitoring,  and  evaluating 
of  performance  on  intellectual  tasks  (Vygotsky,  1978). 

Case  (1984,1985)  argues  that  the  learner’s  ability  to  assemble  increasingly 
sophisticated  executive  control  structures  underlies  most  of  their  cognitive  change. 
According  to  Case,  executive  control  structures  consist  of  three  interrelated  parts:  a 
representation  of  the  problem  situation,  a  representation  of  the  problem  objective, 
and  a  representation  of  the  problem  strategy.  This  tells  us  what  the  structure  of  our 
interaction  with  the  learner  must  contain  to  be  considered  an  executive  control. 

Sternberg  (1984,  1985)  further  refines  the  executive  control  process.  He 
identifies  nine  types  of  metacomponential  processing  that  fulfill  executive 
processing  functions.  Metacomponential  processes  are  subprocesses  of  a  procedural 
skill  much  like  cognitive  strategies  that  help  regulate  the  execution  of  the  procedure. 
Nine  types  of  metacomponential  processing  account  for  many  of  the  consistent 
findings  across  the  literature  on  cognitive  development.  He  contends  that  a  learner 
accomplishing  an  intellectual  task  must: 

•  Determine  the  nature  of  the  problem  under  consideration 

•  Select  the  appropriate  performance  components  for  solving  the  problem 

•  Select  a  strategy  for  combining  performance  components  in  an  optimal 
way 
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•  Select  one  or  more  representations  for  the  information  crucial  to  problem 
solution 

•  Decide  how  to  allocate  processing  resources 

•  Monitor  their  performance 

•  Interpret  feedback  about  the  effectiveness  of  their  efforts 

•  Decide  how  to  act  upon  positive  or  negative  feedback 

•  Modify  their  performance  in  response  to  the  feedback 

Now  that  we  know  exactly  what  the  elements  of  an  effective  executive  control 
strategy  are.  These  elements  must  be  incorporated  into  our  instructional  interaction 
with  the  learner  if  our  objective  is  to  provide  effective  executive  control  strategies. 

Since  studies  have  shown  executive  control  processes  to  be  effective  for 
repairing  bugs  (Burton  &  Brown,  1982;  VanLehn,  1982),  can  they  also  provide  clues 
to  solving  the  problems  associated  with  the  construction  of  fragile  knowledge? 
Referring  back  to  VanLehn’s  most  important  issue  earlier,  “how  is  the  educational 
process  involved  in  the  development  of  bugs?”  Research  on  teacher  behavior  sheds 
some  light  on  the  issue. 

Teacher  Behavior  and  Student  Achievement 


Jere  Brophy  (1979)  conducted  extensive  research  on  teacher  behavior  and  its 
effects  on  student  achievement  to  show  a  significant  correlation  between  teacher 
behavior  and  student  achievement.  Brophy  also  exposes  gaps  in  the  literature  and 
research  methodology.  One  of  the  significant  gaps  is  research  on  the  effectiveness  of 
questioning  techniques  on  student  achievement.  A  synthesis  of  findings  over  several 
studies  indicates  that  factual  questions  with  single  answers  was  correlated  positively 
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and  significantly  with  achievement,  whereas  the  frequency  of  more  complex, 

difficult,  or  divergent  questions  had  negative  correlations  (Brophy,  1979).  The  lack 

of  significant  results  for  complex  or  higher  level  questions  has  puzzled  researchers 

and  caused  them  to  conclude  that  we  need  to  rethink  what  is  meant  by  types  of 

questions  and  their  effects.  Rosenshine  (1976)  summarized  the  findings  of  several 

studies  on  teaching  behaviors  and  student  achievement  conducted  during  the  1970’s 

by  stating  that  “we  are  as  yet  unaware  of  the  optimal  types  and  sequencing  of 

questions... optimal  sequences  may  be  different  for  different  types  of  students  and 

outcomes,  but  we  still  do  not  know  what  these  sequences  are”  (p.  61).  Meredith 

Gall’s  work  at  the  Far  West  Laboratory  for  Educational  Research  and  Development 

supports  Brophy’s  claims.  Gall  (undated,  p.  707)  says  that 

Granting  the  importance  of  questions  in  teaching,  researchers 
still  do  not  know  much  about  them.  What  educational  objectives 
can  questions  help  students  achieve?  What  are  the  criteria  of  an 
effective  question  and  how  can  effective  questions  be  identified? 

Until  researchers  find  answers  to  questions  such  as  these,  a  viable 
behavioral  technology  of  teaching  will  remain  unrealized. 

Gall  (undated,  p.  707)  goes  on  to  say  that  “very  few  researchers  have  explored  the 

relationship  between  teacher’s  questions  and  student  outcomes.” 

Rosenshine  (1976,  p.  61)  confirms  this  by  stating  that  “Since  1940  fewer 

than  25  studies  have  been  conducted  on  any  specific  variable  such  as  teacher  praise 

or  teacher  questions.”  More  recent  research  is  equally  as  sparse.  The  basic 
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conclusion  here  is  that  researchers  agree  that  the  issue  is  important,  but  little  research 
is  actually  being  conducted  in  this  area.  Brophy  and  Gall  support  the  need  for  more 
research  on  teacher  questions.  While  the  need  for  this  research  is  apparent,  research 
on  the  use  of  questioning  as  an  instructional  strategy  is  also  needed. 

Instructional  Interaction  Strategies 

Instructional  interaction  strategies  were  previously  defined  in  Chapter  1 . 
Common  instructional  interaction  strategies  are  addressed  in  the  Minnesota 
Department  of  Education’s  publication  entitled  "Classroom  Instructional  Design: 
Options  for  Teacher/Student  Interaction”  (1989).  This  publication  details  three 
common  teaching  models  for  teacher/student  interaction.  These  interaction  strategies 
are;  directive,  investigative,  and  interactive.  The  directive  strategy  involves  “telling” 
to  present  content.  The  investigative  strategy  involves  a  questioning  approach  to 
achieving  the  lesson  objective,  and  the  interactive  strategy  is  a  mixture  of  both 
telling  and  questioning, 

Perkins  (1992)  hails  Mortimer  Adler’s  (1980)  paieda  proposal  as  a  solution 
to  fragile  knowledge  syndrome.  Adler  proposes  an  interaction  strategy  that  involves 
didactic  instruction  followed  by  coaching  and  Socratic  approaches  to  presenting 
instruction.  Here,  the  didactic  approach  corresponds  to  the  directive  approach  from 
the  Minnesota  group.  Adler’s  coaching  phase  corresponds  to  the  interactive  stage. 
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and  the  Socratic  approach  corresponds  to  the  investigative  strategy  mentioned 
earlier. 


In  the  directive  approach  to  interaction,  the  instructor  or  intelligent  tutoring 
system  is  the  possessor  of  knoAvledge.  This  knowledge  is  presented  to  the  learner  by 
“telling”  or  making  declarative  statements  to  explicitly  direct  the  learner  through  the 
instructional  activities  to  achieve  the  learning  objectives  and  goals.  These  learning 
objectives  could  be  a  complex  process  or  procedure.  The  following  example,  which 
is  the  actual  procedure  to  compute  an  estimated  time  of  arrival  (ETA)  at  a 
predetermined  geographic  point,  illustrates  the  directive  interaction  strategy  in  a 
typical  instructional  scenario: 

Instructor:  “It’s  time  to  compute  an  ETA.  The  first  thing  you  need 
to  do  to  compute  an  ETA  is  to  get  the  aircraft’s 
groundspeed  from  the  Navigation  Computer  System. 

Then,  take  your  plotter  and  measure  from  the  fix  you 
resolved  and  plotted  on  your  chart  to  the  geographic 
point  that  you  would  like  to  compute  the  ETA  to. 

Now,  divide  the  distance  you  measured  with  your  plotter 
by  the  groundspeed.  The  result  is  how  long  it  will  take 
to  fly  the  distance  you  measured.  Add  this  time  to  the 
time  of  your  fix  and  you  now  have  your  estimated 
time  of  arrival.” 

At  this  point  in  the  scenario  the  instructor  would  then  repeat  the  same  instruction 
once  again  as  the  student  performed  the  task. 

In  the  questioning  approach  to  interaction,  questions  are  used  to  lead  the 
learner  through  the  accomplishment  of  the  process.  Making  declarative  statements  to 
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the  learners  is  not  part  of  this  interaction  strategy.  The  questioning  approach  should 
not  be  confused  with  inquiry  teaching,  because  in  this  case,  what  is  to  be  learned  is  a 
previously  established  process  that  has  a  correct  sequence  for  accomplishment  and 
consequences  for  incorrect  sequences.  Learners  are  not  encouraged  to  invent  their 
own  sequences  or  cases  and  test  their  hypotheses.  Again  the  same  task  is  used  to 
illustrate  the  questioning  approach: 

Instructor:  “What  do  we  need  to  do  right  now?” 

Student:  “We  need  to  compute  an  ETA  to  our  destination.” 

Instructor:  “How  do  we  do  that?” 

Student:  “First,  we  need  to  get  the  aircraft’s  groundspeed.” 

Instructor:  “Where  do  we  get  that  information?” 

Student:  “We  get  it  from  the  Navigation  Computer  System.” 

Instructor:  “What  do  we  do  with  the  groundspeed?” 

(The  exchange  then  continues  until  the  student  has  properly  and  correctly  computed 
an  ETA). 

In  the  mixed  approach  to  instructional  interaction,  learners  are  first  told  how 
to  accomplish  the  process  (as  in  the  directive  approach)  and  then  questioned  through 
it  as  they  perform  it  (as  in  the  questioning  approach).  Remember,  each  of  these 
strategies  involve  various  ways  of  interacting  with  the  learner  to  present  a  previously 
established  procedure.  The  learner’s  ability  to  perform  the  process  correctly  and 
autonomously  indicates  successful  acquisition  of  the  procedural  skill  prescribed  by 
the  learning  objective. 
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All  three  interaction  strategies  mentioned  previously  represent  the  mean  and 
the  extremes  for  common  instructional  interaction  strategies  within  the  context  of 
procedural  skill  acquisition.  So,  which  of  these  strategies  is  most  effective  for 
reducing  fragile  knowledge  construction?  Are  some  strategies  more  effective  than 
others  in  eliminating  or  reducing  the  construction  of  particular  types  of  fragile 
knowledge? 

Brophy  (1979)  and  Good  (1979)  conclude  that  direct  instruction  has 
important  advantages  and  is  clearly  superior  to  other  approaches  for  producing 
mastery  of  basic  skills.  Advocates  of  questioning  approaches  point  to  the  ability  to 
encourage  learners  to  reflect  and  thus  stimulate  thinking  and  learning  of  the  process 
or  procedural  skill  being  learned  (Aschner,  1961).  Yet,  Adler  (1980)  and  Perkins 
(1992)  argue  that  the  best  interaction  strategy  is  to  provide  a  didactic  and 
questioning  approach  together  as  a  congruent  instructional  interaction  strategy.  Who 
is  right?  Does  the  interaction  strategy  even  matter  at  all?  The  purpose  of  this  study  is 
to  find  an  answer  to  these  questions. 

Hypotheses 

This  experiment  examined  the  effect  of  common  instructional  interaction 
strategies  on  the  construction  of  fragile  knowledge  in  learners.  Furthermore,  the 
experiment  assessed  whether  providing  learners  with  executive  control  strategies  can 
inhibit  the  production  of  fragile  knowledge  during  initial  procedural  skill 
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acquisition.  Prominent  educational  theorists  and  researchers  hold  conflicting 
positions  on  the  efficacy  of  various  instructional  interaction  strategies  to  produce 
rich  understanding  (and  less  fragile  knowledge)  in  the  learner.  This  study  examined 
each  of  these  positions  and  tested  their  effectiveness  in  eliminating  or  reducing  the 
construction  of  fragile  knowledge. 

Hypothesis  1  The  Effect  of  Interaction  Strategies  on  Fragile  Knowledge 
Construction 

Learners  who  receive  minimal  or  directive  type  strategies  will  construct  more 
fragile  knowledge  than  learners  who  received  more  facilitative  (questioning  or 
mixed)  styles.  With  respect  to  the  first  independent  variable,  instructional  interaction 
strategies,  which  had  four  levels:  no  instructional  interaction,  directive,  questioning, 
and  mixed  (directive  and  questioning),  the  results  of  the  experiment  will  show  that 
the  interaction  strategy  makes  a  significant  difference  on  the  learner’s  construction 
of  fragile  knowledge.  This  will  be  indicated  by  the  no  instructional  interaction  and 
directive  groups  presenting  lower  total  scores  on  the  posttest.  The  questioning  and 
mixed  groups  will  present  higher  scores  on  the  posttest. 

Hypothesis  2  The  Efficacy  of  Executive  Control  Strategies 

Regarding  the  second  independent  variable,  executive  controls,  which  had 
two  levels:  the  explicit  teaching  of  executive  control  strategies  and  no  explicit 
teaching  of  executive  control  strategies,  the  results  of  the  experiment  will  show  that 
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possessing  and  using  executive  control  strategies  will  minimize  the  construction  of 
fragile  knowledge.  This  will  be  indicated  by  higher  scores  on  the  posttest  than  the 
corresponding  group  that  did  not  receive  explicit  instruction  on  executive  control 
strategies.  For  example,  the  directive  with  executive  controls  group  will  achieve 
higher  scores  on  the  posttest  than  the  directive  without  executive  controls  group  and 
the  same  will  follow  for  each  of  the  remaining  treatment  groups. 

Hypothesis  3  The  Interaction  of  Instructional  Interaction  Strategy  and  Control 
Processes 

Concerning  the  issue  of  possible  interaction  between  the  two  independent 
variables,  the  results  of  this  experiment  will  also  show  that  there  is  no  significant 
interaction  between  instructional  interaction  strategies  and  the  explicit  teaching  of 
executive  control  processes.  Thus,  there  will  be  no  particular  combinations  of 
instructional  interaction  strategies  and  the  absence  or  presence  of  executive  controls 
that  will  produce  statistically  different  results  on  the  construction  of  fragile 
knowledge. 

The  theory  contained  in  the  literature  of  this  chapter  indicates  that  the 
interaction  strategy  should  in  fact  matter.  The  theoretical  argument  is  that  encoding 
of  condition-action  productions  takes  place  during  the  associative  stage  of 
procedural  skill  acquisition.  In  keeping  with  the  information-processing  model, 
different  knowledge  compilers  should  produce  different  encoding  of  condition- 
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action  productions.  Input  from  domain  experts  to  the  compiler  or,  ultimately,  the 
compilation  process  should  produce  encoded  condition-action  productions  that  are 
more  stable  and  less  tenuous  than  unassisted  knowledge  compilation.  Thus,  assisted 
compilation  should  be  less  fragile  or  contain  less  fragile  knowledge.  Furthermore, 
encoding  executive  control  processes  along  with  the  new  procedural  skill  should 
provide  learners  with  more  tools  to  resolve  impasses  or  repair  bugs.  This  ability  to 
autonomously  resolve  these  impasses  or  bugs  allows  learners  access  to  knowledge 
not  previously  attainable  and  stabilizes  what  was  previously  fragile  knowledge.  The 
research  design,  methodology,  and  material  reflect  the  assertions  posed  in  this 
chapter  and  help  to  ensure  meaningful  and  useful  results. 
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CHAPTERS 


EXPERIMENTAL  DESIGN,  METHODOLOGY,  AND  MATERIALS 

Methodology 

This  study  examined  the  relative  effects  of  four  common  instructional 
interaction  strategies:  no  instructional  interaction,  directive,  inquiry  or  questioning, 
and  mixed  (directive  and  questioning)  strategies  along  with  the  presence  or  absence  of 
explicit  instruction  on  executive  control  processes  and  their  effects  on  the 
construction  of  fragile  knowledge  during  initial  procedural  skill  acquisition.  Subjects 
were  instructed  in  five  basic  navigation  procedural  skills  according  to  the  interaction 
strategy  prescribed  by  their  respective  treatment  group.  The  five  basic  navigation 
procedural  skills  are  listed  here  in  Table  3.1: 

Table  3.1 

Basic  Navigation  Procedural  Skills 

Determining  an  initial  heading  to  fly  to  navigate  to  a  prescribed  geographic  point 
Determining  the  aircraft’s  actual  position  using  a  radio  aid  to  navigation 
Determining  the  aircraft’s  actual  position  using  the  RADAR 
Computing  a  heading  to  alter  the  aircraft  to  when  off  the  desired  course 
Computing  an  Estimated  Time  of  Arrival  (ETA) 

Determining  the  aircraft’s  most  probable  position  in  the  future 
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This  chapter  provides  a  detailed  description  of  the  participants,  design, 
experimental  materials,  instrumentation,  experimental  procedures,  and  evidence  of 
compliance  with  the  assumptions  for  the  statistical  analyses  conducted  in  this  study. 

Participants 

One  hundred  and  eighty-four  freshmen  undergraduate  students  enrolled  in  an 
introductory  aviation  science  course  participated  in  this  study.  AVI 00  (Introduction 
to  Aviation)  is  a  required  course  for  all  first-year  cadets  at  the  United  States  Air 
Force  Academy.  This  study  was  conducted  during  the  Fall  1995  semester.  There  were 
approximately  159  males  and  25  females,  ranging  in  age  from  17  to  20.  The  students 
entering  the  Academy  are  selected  from  the  applicants  who  typically  score  in  the  top 
twenty-five  percent  on  the  SAT/ACT  exams.  The  average  SAT  score  was  1430.  All 
students,  regardless  of  academic  major,  are  required  to  complete  the  core  curriculum 
at  the  Academy.  The  core  curriculum  focuses  on  hard  science  and  general 
engineering.  No  subject  selected  for  this  study  had  any  formal  navigation  training.  All 
subjects  completed  the  Academy’s  summer  training  and  orientation  programs.  The 
Academy  environment  for  first-year  cadets  is  very  regimented,  strict,  and  demanding. 
Twenty-one  credit  hours  with  several  concurrent  extracurricular  activities  is  a  typical 
load  at  the  Academy.  Consequently,  first-year  cadets  are  used  to  a  highly 
proceduralized  learning  and  living  environment  and  are  compliance  oriented  toward 
authority.  Summer  training  and  orientation  programs  acclimated  the  cadets  to  the 
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militaiy  instructional  environment  before  the  study  began.  They  also  completed  the 
introductory  microcomputer  orientation  for  first-year  cadets.  The  orientation 
provided  the  subjects  with  specific  microcomputer  training  on  hardware,  start-up 
procedures,  keyboarding  and  use  of  the  mouse,  operating  systems,  operation  of  the 
local  area  network,  and  word  processing.  At  the  time  this  study  was  conducted, 
subjects  had  already  completed  assignments  on  their  computers  for  other  classes  such 
as  English,  history,  physics  or  chemistiy  which  was  evidence  of  their  proficiency  with 
the  same  microcomputer  system  that  the  simulator  is  based  on.  The  students’  reading 
skills  were  tested  during  inprocessing  to  the  Academy  and  were  at  least  an  1 1.5 
reading  level  on  the  SRA  reading  scale  for  both  speed  and  comprehension. 

Student  participation  in  this  study  was  part  of  their  normal  core  academic 
schedule,  specifically  for  this  Aviation  Science  class.  A  pretest  assessed  their  level  of 
prior  knowledge  of  basic  navigation.  The  pretest  was  a  parallel  form  of  the  posttest. 
The  test  covered  the  same  basic  air  navigation  principles  and  procedures  as  the 
posttest.  The  purpose  of  the  pretest  was  to  assess  the  amount  of  prior  knowledge  the 
subject  possessed  prior  to  the  treatment.  Any  subject  who  scored  above  80%  was 
considered  to  have  a  significant  amount  of  prior  knowledge  and  was  not  selected  for 
participation  in  the  study.  The  eighty  percent  cut  score  on  the  pretest  was  established 
at  the  recommendation  of  Academy  institutional  research  personnel  who  have 
conducted  numerous  studies  on  Academy  cadets.  Academy  cadets  are  very  high 
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ability  subjects  who  are  highly  motivated  in  this  particular  subject  area.  However,  no 
subjects  were  actually  eliminated  from  the  study  due  to  a  significant  amount  of  prior 
knowledge  because  the  highest  score  on  the  pretest  was  76%.  The  participants  were 
randomly  assigned  to  a  class  section  and  each  class  section  was  randomly  assigned  to 
one  of  eight  different  treatment  groups.  Ten  sections  out  of  a  possible  sixty  sections 
were  selected  to  participate  in  this  study.  Each  section  contained  approximately  20 
subjects  who  were  randomly  assigned  by  computer  to  that  particular  section.  The 
subjects  were  motivated  by  the  typical  pressures  of  a  competitive  academic 
environment  and  special,  additional,  highly-desired  incentive  activities  for  those  that 
distinguished  themselves  by  their  performance  on  the  pre  and  posttest  instruments. 
The  tests  used  in  the  study  were  part  of  their  regular  academic  program  in  AV-lOO 
and  counted  toward  their  class  standing.  Class  standing  determined  who  received  the 
additional  incentive  activities. 

Design 

The  design  of  the  experiment  was  a  4  X  2  factorial  design  (see  Table  3.2 
below)  in  which  the  effects  of  the  absence  or  presence  of  executive  control  strategies 
within  four  different  instructional  interaction  strategies  were  tested  for  their  effects  on 
the  amount  of  fragile  knowledge  constructed  during  initial  procedural  skill 
acquisition. 
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Table  3.2 


Experimental  Design 


Instructional  Interaction  Strategy 


No  Interaction 

Directive 

Questioning 

Mixed 

Executive 

Controls 

Group  1 

Group  3 

Group  5 

Group  7 

No  Executive 
Controls 

Group  2 

Group  4 

Group  6 

Group  8 

The  experimental  design  followed  Campbell  and  Stanley’s  (1963)  pretest-posttest, 

control  group  design.  The  graphical  depiction  of  the  experiment  is: 

ROX,  O 
ROX2O 
R  O  X3  O 
ROX4O 
ROXjO 
ROXfiO 
ROX7O 
RO  O 


The  first  independent  variable  was  the  instructional  interaction  strategy.  These 
strategies  included;  no  instructional  interaction,  directive,  questioning,  and  Mixed 
(directive  and  questioning)  interaction  strategies.  The  second  independent  variable 
was  absence  or  presence  of  explicit  instruction  on  executive  control  processes  to 
double-check  work  or  provide  a  broader  and  deeper  understanding  of  the  concepts 
and  principles  underlying  the  procedures. 
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Eight  treatments  were  used  in  this  experiment.  Each  of  these  treatments  was 
composed  of  combinations  of  the  two  independent  variables  and  was  depicted  earlier 
in  Table  3.2  entitled  “Experimental  Design.” 

Variables  Studied 

The  relative  effects  of  two  independent  variables  were  studied.  The  first 
independent  variable  was  the  interaction  strategy  used,  with  four  levels.  The  second 
independent  variable  was  the  presence  versus  absence  of  explicit  instruction  on 
executive  control  processes.  This  independent  variable  had  two  levels. 

Independent  Variables 

Instructional  Interaction  Strategy.  The  effect  of  varying  the  instructional 
interaction  strategy  (one  independent  variable  with  four  levels)  was  the  first 
independent  variable  studied  (see  Table  3.2).  This  experiment  tested  common 
interaction  strategies.  These  common  instructional  interaction  strategies  were  no 
instructional  interaction  (the  control  group),  directive  strategies,  questioning 
strategies,  and  mixed  (directive  and  questioning)  strategies. 

Executive  Control  Processes.  The  second  independent  variable  studied  was 
the  effect  of  explicit  instruction  on  executive  control  processes  (one  independent 
variable  with  two  levels).  Perkins  (1985)  pointed  to  direct  instruction  on  executive 
processes  as  a  significant  inhibitor  of  fragile  knowledge  construction.  Eight  treatment 
groups,  as  described  previously,  tested  this  posit. 
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Dependent  Variables 


The  dependent  variables  were  the  amount  and  types  of  fragile  knowledge  that 
were  constructed  as  a  result  the  instructional  interaction  strategy  the  subject  received. 
The  dependent  measure  was  a  multiple-choice  immediate  posttest  given  at  the 
conclusion  of  the  treatment  to  assess  the  amount  of  fragile  knowledge  the  subject 
possessed.  The  test  represented  a  solid  grasp  of  navigational  concepts,  principles,  and 
procedures.  A  low  score  on  the  measure  indicated  the  presence  and  degree  of  fragile 
knowledge  present  in  the  subject.  A  detailed  description  of  the  posttest  is  provided 
under  the  discussion  of  the  individual  instruments. 

Treatments 

There  were  eight  treatment  groups  (see  Table  3.2  earlier  in  this  chapter). 
Groups  1  and  2,  groups  3  and  4,  groups  5  and  6,  and  groups  7  and  8  differed  by  the 
instructional  interaction  style  or  strategy  they  received.  Each  odd  numbered  group 
additionally  differed  by  the  presence  or  absence  of  explicit  instruction  on  executive 
control  processes  that  could  be  used  to  cross-check  or  provide  a  better  understanding 
of  the  air  navigation  process. 

Treatment  groups  1  and  2  did  not  receive  any  instructional  interaction.  Group 
1  was  provided  information  on  executive  control  processes  by  additional  text 
covering  these  subjects.  Group  2  was  the  control  group.  This  treatment  group  was 
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not  provided  any  instructional  interaction  or  the  additional  coverage  on  executive 
control  processes  in  its  text. 

Treatment  groups  3  and  4  were  given  directive  or  didactic  instruction  on 
navigation  principles  and  procedures.  Subjects  in  this  group  were  told  what  to  do  and 
how  to  do  it  from  the  beginning  until  the  end  of  the  simulator  mission.  Group  3  was 
given  additional  instruction  on  how  to  cross-check  their  work,  identify  and  correct 
their  own  errors,  and  orient  themselves  strategically.  Group  4  was  not  provided  with 
that  instruction. 

Groups  5  and  6  experienced  a  more  facilitative  instructional  strategy. 
Instructors  in  this  group  used  questions  instead  of  making  statements  to  lead  subjects 
through  the  procedures.  Instructors  provided  subjects  in  the  5th  group  with  the  key 
questions  they  needed  to  ask  themselves  to  prevent  or  correct  errors  and  orient 
themselves.  Group  6  did  not  receive  this  instruction. 

Groups  7  and  8  received  a  combination  of  interaction  strategies  from  the  two 
previous  groups.  Instructors  used  statements  and  questions  to  instruct  subjects  in  this 
group.  Once  again,  subjects  in  Group  7  were  given  explicit  instruction  on  strategies 
and  executive  control  processes  to  facilitate  autonomous  performance. 

Materials 

The  primary  device  used  for  the  experiment  was  a  microcomputer-based 
simulation  of  the  navigator  station  of  an  Air  Force  T-43  A.  The  T-43  is  a  military 
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version  of  the  Boeing  Til -200  commercial  airliner.  The  NAVSIM  (navigation 
simulator)  graphically  reproduces  the  panel  in  the  aircraft  and  is  supported  by  the 
actual  flight  model  for  the  Boeing  737  resulting  in  simulation  that  is  realistic  enough 
to  reduce  the  number  of  inflight  training  missions. 

NAVSIM  runs  on  an  IBM  PC  based  microcomputer  and  requires  at  least  an 
80386  microprocessor,  I  MB  RAM,  1  floppy  disk  drive,  and  an  EGA  monitor.  The 
actual  system  used  for  the  experiment  was  an  80486DX/66  MHZ  PC  with  8MB 
RAM,  120B  hard  drive,  and  an  SVGA  monitor.  The  NAVSIM  has  been  used  to  train 
students  and  instructors  in  the  T-43  for  the  past  five  years.  The  software  is  updated 
regularly  to  accurately  reflect  changes  to  the  aircraft’s  systems  and  performance. 

The  other  materials  used  were  standard  Air  Force  issue  items.  These  items 
included  a  current  aeronautical  navigation  chart  for  the  route  to  be  flown,  a 
navigation  plotter,  a  Dead  Reckoning  Computer,  a  current  IFR  Supplement,  and  a 
navigation  worksheet. 

Topic  Selection 

Three  fundamental  concerns  prompted  the  selection  of  the  subject.  The  first 
factor  in  the  topic  selection  was  the  concern  to  address  a  fundamental,  authentic, 
problem.  Fragile  knowledge  can  have  dire  consequences.  Educational  professionals 
have  an  ethical  responsibility  to  address  the  issue  of  fragile  knowledge  and  any 
influence  they  might  have  on  the  construction  of  it  in  the  products  they  produce  or  the 
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people  they  instruct.  The  second  factor  was  the  need  to  teach  a  new  procedural 
subject  to  motivate  the  learners.  Teaching  a  new  subject  reduced  possible 
contamination  from  prior  general  knowledge  or  experience.  The  need  for  a  large 
number  of  subjects  was  the  final  factor  in  the  selection  of  this  particular  topic  for  the 
study.  A  sufficiently  large  sample  was  required  to  cover  the  range  of  common 
interaction  strategies,  examine  the  effect  of  strategic  controls  within  these  groups,  and 
retain  statistical  power.  Navigation  is  a  good  candidate  because  all  first-year  students 
at  the  Air  Force  Academy  are  required  to  complete  the  course  to  graduate. 

Once  the  academic  subject  area  was  established,  ten  subject  matter  experts 
reviewed  the  course  content  and  objectives  to  establish  the  content  validity  of  the 
instructional  materials.  The  Subject  Matter  Experts  identified  five  primaiy  procedural 
skills  used  in  basic  air  navigation  (see  Table  3.1).  These  procedures  and  their 
underlying  concepts  and  principles  were  taught  lecture  style  in  the  first  four  lessons  of 
the  course  (eight  contact  hours).  This  experimental  treatment  used  the  actual  training 
methods  and  materials  the  Air  Force  Academy  uses  to  teach  flight  training  students  to 
navigate  the  T-43.  Only  the  instructional  interaction  strategy  was  changed  for  this 
study;  the  goals,  objectives,  content,  methods,  or  materials  were  not  altered  for  the 
experiment. 
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Instruments 


Pretest  and  Posttest 

These  two  tests  were  instructor  developed  multiple  choice  examinations 
designed  to  measure  the  subjects’  ability  to  perform  the  five  basic  navigational 
procedures  (see  Table  3.1)  and  test  their  understanding  of  the  concepts  and  principles 
underlying  the  procedures. 

Each  test  contained  21  multiple  choice  questions  with  an  additional  29  fill-in- 
the-blank  substeps  (see  Appendices  D  and  E).  Each  question  required  the  subject  to 
demonstrate  his  or  her  ability  to  perform  an  entire  process  or  a  part  of  one  of  the  five 
basic  navigation  procedures  mentioned  earlier.  To  answer  these  questions  subjects 
had  to  refer  to  print  screens  from  the  simulator  which  were  included  as  supplemental 
material  in  the  test.  This  provided  graphical  depictions  of  real-world  conditions  which 
the  subject  then  had  to  interpret  just  as  they  would  inflight.  Each  question  had  three 
or  four  possible  answers,  one  correct  answer  and  two  or  three  distracters.  The 
distracters  were  based  on  fragile  knowledge  of  navigational  concepts,  principles,  or 
procedures.  The  lower  the  subjects’  score  on  the  instrument  the  higher  degree  of 
fragile  knowledge  the  subject  possessed. 

The  pretest  and  posttest  were  initially  developed  by  a  highly  experienced 
instructor.  Ten  subject  matter  experts  then  reviewed  the  instruments  for  their  content 
validity,  comprehensiveness,  and  accuracy.  The  ten  subject  matter  experts  also 
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reviewed  the  distracters  in  each  question  to  ensure  that  these  distracters  represented 
valid  examples  of  the  common  types  of  fragile  knowledge  as  defined  in  the  definitions 
section  of  chapter  1.  The  tests  were  changed  based  on  input  from  the  subject  matter 
experts  until  a  consensus  was  reached  among  the  ten  subject  matter  experts.  Parallel 
forms  were  used  to  reduce  testing  effects  and  to  ensure  the  security  and  integrity  of 
the  instruments.  Recent  graduates  of  the  course  were  used  to  conduct  a  formative 
evaluation  on  both  instruments.  The  formative  evaluation  group  took  both  the  pretest 
and  the  posttest  during  the  same  testing  session.  Results  of  the  formative  evaluation 
were  used  to  improve  the  form  and  the  function  of  the  pre  and  posttest  instruments. 
An  item  analysis  showed  the  parallel  forms  reliability  to  be  .88. 

Procedure 

Pilot  Study 

Once  the  instrumentation  was  finished,  I  conducted  a  pilot  study.  I  randomly 
selected  eight  additional  recent  graduates  to  participate  in  the  pilot  study.  These 
participants  were  randomly  assigned  to  one  of  four  possible  treatment  groups  in  the 
pilot  study.  The  materials  and  procedures  mirrored  the  main  study.  Results  from  the 
pilot  study  were  generally  in  the  hypothesized  directions.  Only  minor  changes  were 
made  as  a  result  of  the  pilot  study.  These  changes  generally  focused  on  clarifying 
verbiage  in  the  instructions  given  to  the  subjects  during  the  simulator  mission. 
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The  experimental  procedure  followed  a  pretest,  intervention,  and  posttest 
sequence.  The  pretest  was  administered  immediately  after  the  regular  academic 
portion  of  the  course.  The  treatment  was  conducted  as  two  instructional  practice 
simulator  missions.  The  immediate  posttest  was  a  mid-course  examination. 
Pre-Treatment  Training  of  Instructors 

Variability  among  the  sixty  instructors  who  participated  in  this  study  was  a 
significant  concern.  Several  steps  were  taken  to  minimize  this  possible  effect.  First, 
instructors  were  sent  an  overview  of  the  study  with  sufficient  details  explaining  the 
purpose,  scope,  methodology,  and  procedures  of  the  research  effort.  I  then  held  a 
meeting  with  the  instructors  to  provide  direct  interaction  and  answer  any  questions 
before  instructor  training  commenced. 

Instructor  training  consisted  of  a  one  hour  and  forty  minute  practice  teaching 
simulator  mission.  I  gave  each  instructor  a  job  aid  to  remind  them  of  how  to  conduct 
their  interaction  with  the  subject  according  to  their  treatment  group  and  a  checksheet 
to  remind  the  instructor  of  the  content  that  needed  to  be  covered  during  the  lesson. 
During  the  practice  mission  on-the-spot  corrections  and  constructive  feedback  were 
given  when  instructors  deviated  from  the  prescribed  interaction  strategy. 

During  the  actual  treatment,  instructors  used  the  same  job  aids  from  the 
practice  mission  to  remind  them  of  their  interaction  strategy  roles.  I  continued  to 
provide  on-the-spot  corrections  to  the  offending  instructors  when  they  deviated  from 


47 


their  prescribed  roles.  If  I  had  to  provide  more  than  one  correction  to  an  instructor  I 
rejected  his  or  her  data. 

Classroom  Pretest 

I  administered  the  pretest  in  the  classroom  prior  to  the  start  of  the  fifth 
classroom  lesson  on  navigation  principles.  By  this  point,  subjects  had  attended  eight 
contact  hours  of  classroom.  The  pretest  was  a  parallel  form  of  the  posttest.  The 
pretest  took  twenty-two  minutes  for  the  quickest  student  and  thirty-one  minutes  for 
the  slowest  student.  The  classroom  instructor  told  the  subjects  that  the  test  would 
count  toward  their  class  standing  and  opportunity  for  incentive  flights.  No  one  gave 
the  subjects  any  feedback  on  how  they  performed  on  the  pretest.  Furthermore,  they 
were  restricted  from  discussing  anything  about  the  test  with  anyone  else  or  going 
back  and  reviewing  any  of  the  content  areas  on  the  test  until  after  completing  the 
second  simulator  mission  (which  included  the  posttest). 

Assignment  to  Treatment 

The  participants  were  randomly  selected  and  randomly  assigned  to  one  of 
eight  treatment  groups.  Here’s  how  this  was  accomplished.  First,  the  Registrar 
segmented  the  course  offering  into  six  modules.  Each  module  contained  ten  sections 
(one  section  contained  approximately  20  students).  Three  modules  were  offered  per 
semester,  and  each  module  occurred  one  after  the  other.  Then,  the  scheduling 
computer  randomly  assigned  each  subject  to  one  of  the  sixty  possible  sections  of  AV- 
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100  within  these  modules.  The  computer  did  this  by  alphabetically  assigning  each 
participant  to  the  first  available  section  until  the  section  was  full.  I  conducted  this 
study  during  the  fall  semester;  so,  that  left  three  modules  to  choose  from.  Due  to 
coordination  problems  with  the  department,  I  wasn’t  able  to  conduct  the  study  until 
late  in  the  semester.  The  coordination  problem  limited  my  access,  and  effectively 
made  the  choice  of  which  module  to  study  for  me.  I  got  together  with  the  Course 
Director  and  we  each  picked  random  numbers  from  one  to  ten  to  assign  each  of  the 
ten  sections  within  the  module  to  one  of  the  eight  treatment  groups. 

Instructional  Intervention 

I  gave  the  subjects  the  pretest  when  they  arrived  for  their  first  simulator 
mission.  The  instructors  started  the  simulator  mission  once  all  the  subjects  had 
finished  the  pretest.  Each  group  had  approximately  twenty  students.  Additional 
instructors  were  used  to  reduce  the  student  to  instructor  ratio  to  4:1.  All  simulator 
missions  started  at  the  same  prescribed  geographical  point,  flew  a  prescribed  route, 
and  ended  at  an  established  termination  point.  Only  the  instructional  interaction  style 
and  instruction  on  executive  control  processes  varied  by  treatment  group. 

Subjects  started  the  second  simulator  mission  immediately  after  they  arrived. 
The  second  simulator  mission  was  identical  to  the  previous  one.  Instructors 
administered  the  posttest  when  the  subjects  either  completed  the  mission  or  could  no 
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longer  continue  the  mission.  The  instructors  thanked  and  released  the  participants 
once  they  completed  the  posttest. 

Data  Collection 

I  handscored  the  multiple-choice  pre  and  posttests  to  prevent  errors  due  to 
machine  misfeeds  or  the  mismarking  of  answer  sheets.  I  entered  student  responses  to 
each  question  along  with  pretest  and  posttest  scores  directly  into  the  SPSS  Data 
Editor.  This  served  as  the  master  database  for  all  statistical  analyses.  This  database 
also  contained  the  pre-study  data  collected  earlier.  All  simulator  missions  were 
recorded  on  3.5”  floppy  diskettes.  Every  action  the  subject  did  during  the  entire 
mission  was  recorded  for  future  reference  and  could  be  played  back  for  analysis. 
Data  Analysis 

SPSS  for  Windows  6.1,  Statistical  Package  for  the  Social  Sciences 
(SPSS)(1994),  a  microcomputer  based  statistics  package  was  used  on  an  IBM  based 
80486  microcomputer,  to  conduct  all  statistical  analyses.  All  data  were  entered 
directly  into  the  program’s  data  editor. 

I  performed  an  Analysis  of  Covariance  (ANCOVA)  on  the  total  score 
obtained  on  the  posttest,  with  the  total  score  obtained  on  the  pretest  as  the  covariate. 
The  categories  of  the  4  X  2  factorial  design  were  instructional  interaction  style  by 
executive  control  process.  Also,  I  conducted  a  Multivariate  Analysis  of  Variance 
(MANCOVA)  to  determine  whether  specific  instructional  interaction  strategies 
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produced  particular  types  of  fragile  knowledge.  I  did  this  by  recoding  each  student 
response  to  every  question  on  the  posttest  into  one  of  the  five  types  of  fragile 
knowledge  that  the  response  represented.  For  example,  if  a  subject  selected  answer  B 
to  question  number  one  and  the  subject  matter  experts  had  previously  determined  that 
answer  B  was  a  valid  example  of  inert  knowledge,  then  the  subject’s  response  was 
recoded  into  one  instance  of  a  display  of  inert  knowledge.  For  the  MANCOVA,  the 
instructional  interaction  strategy  was  the  factor  variable  and  the  five  types  of  fragile 
knowledge  (inert,  partial,  etc.)  were  the  dependent  variables.  As  a  result  of  the 
recoding,  these  dependent  variables  contained  the  number  of  instances  each  subject 
displayed  that  particular  type  of  fragile  knowledge  on  the  posttest.  An  alpha  level  of 
.05  was  used  for  all  statistical  analyses.  A  discussion  and  evidence  of  compliance  with 
the  assumptions  necessary  for  conducting  a  valid  analysis  of  variance  and  multivariate 
analysis  of  variance  follows. 

Assumptions 

The  assumptions  for  analysis  of  covariance  are  ;  (1)  random  and  independent 
sampling,  (2)  samples  from  a  normal  distribution,  (3)  true  variances  of  each  group  are 
equal,  (4)  independence  of  means  and  variances,  (5)  at  least  interval  scaling  (Gabriel 
&  Hopkins,  1974,  p.  378),  (6)  a  correlation  between  the  covariate  and  the  dependent 
variable,  and  (7)  homogeneity  of  regression  (Keppel  &  Zedeck,  1989).  The  students 
were  randomly  assigned  to  groups  according  to  the  plan  presented  earlier  in  this 
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chapter.  No  subjects  were  assigned  to  more  than  one  treatment  group.  The  number  of 
participants  in  each  treatment  group  varied  from  1 1  to  3 1  in  the  final  statistical 
analysis.  Stem  and  Leaf  Plot  analysis  of  posttest  scores  revealed  that  students  fulfilled 
the  requirements  for  a  normal  distribution.  Additionally,  posttest  scores  further 
indicated  that  each  group  followed  a  normal  curve  centered  around  a  58.8%  mean 
with  an  1 1.8%  standard  deviation.  Cochran’s  C  and  Bartlett-Box  homogeneity  of 
variance  tests  conducted  on  the  posttest  data.  Cochran’s  C(17,8)=.14569  p=1.000 
and  Bartlett-Box(7, 14,688)=. 06243  p=1.000  supported  the  null  hypothesis  that  all 
cell  variances  were  equal.  Data  in  Tables  3.2  and  4.1  support  the  independence  of 
means  and  variances.  Interval  scaling  was  assumed  by  constructing  the  posttest  so 
that  scores  ranged  from  0  to  100.  Correlation  between  the  covariate  and  the 
dependent  variable  is  established  by  the  fact  that  the  pretest  (covariate)  was  a  parallel 
form  of  the  posttest.  Independence  was  achieved  by  taking  the  pretest  measure  before 
the  treatment  occurred.  Homogeneity  of  regression  was  verified  by  examining  the 
residual  plots  for  each  treatment  group.  All  plots  showed  only  minor  deviations  from 
linearity.  The  deviations  between  the  pretest  residuals  and  posttest  residuals  were  in 
the  same  direction  and  nearly  the  same  amounts. 

The  assumptions  for  multivariate  analysis  of  variance  are  an  extension  of  the 
assumptions  for  univariate  analysis  of  variance.  As  mentioned  previously,  the 
assumptions  for  analysis  of  covariance  are  ;  (1)  random  and  independent  sampling,  (2) 
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samples  from  a  normal  distribution,  (3)  true  variances  of  each  group  are  equal,  (4) 
independence  of  means  and  variances,  (5)  at  least  interval  scaling  (Gabriel  &  Hopkins, 
1974),  (6)  a  correlation  between  the  covariate  and  the  dependent  variable,  and  (7) 
homogeneity  of  regression  (Keppel  &  Zedeck,  1989).  These  assumptions  still  hold  for 
multivariate  analysis  with  the  added  assumption  that  the  composite  observations  are 
normally  distributed,  equally  variable  in  the  populations  sampled,  and  independent 
(Gabriel  &  Hopkins,  1974).  The  data  used  in  this  analysis  were  the  same  data  used  in 
the  previous  univariate  analysis.  Posttest  responses  were  recoded  to  represent  the 
type  of  fragile  knowledge  indicated  by  the  answer  the  subject  selected  during  the 
posttest  measurement.  Since  these  assumptions  have  already  been  met,  they  will  not 
be  addressed  again.  However,  compliance  with  the  additional  assumption  posed  by 
the  multivariate  analysis  has  not  yet  been  demonstrated,  and  a  brief  examination 
follows. 

The  assumption  of  normal  distribution,  equal  variance,  and  independence  of 
composite  observations  for  the  multivariate  analysis  was  not  met  for  the  recoded  data. 
Analysis  of  stem  and  leaf  plots  for  the  five  types  of  fragile  knowledge 
(conglomerative,  inert,  misplaced,  naive,  and  partial)  showed  a  normal  distribution  for 
all  types  except  inert  knowledge.  The  inert  category  also  violated  the  homogeneity  of 
variance  assumption  with  a  Cochran’s  C(3 6,4)=. 62344  and  p<. 0001.  Multivariate 
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tests  for  homogeneity  of  dispersion  were  also  violated.  These  tests  revealed  a  Box’s 
M  of  F=157.3,  df(45, 30425),  p<.0001. 

Although  multivariate  assumptions  for  all  five  categories  were  not  met  due  to 
the  effect  of  the  inert  category,  assumptions  for  the  remaining  four  categories  were 
met.  Since  the  assumptions  for  the  remaining  categories  were  met,  a  MANCOVA  was 
conducted  with  the  inert  category  removed  from  the  analysis.  The  PRETEST  was 
used  as  a  covariate  to  account  for  aptitude  effects  and  keep  this  analysis  consistent 
with  the  ANCOVA  for  the  main  effects  of  the  first  research  question. 

With  the  assumptions  for  the  analyses  met,  the  analyses  were  completed,  and 
the  results  of  the  study  follow. 
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CHAPTER  4 


RESULTS 

This  chapter  presents  the  results  of  the  statistical  analysis  of  the 
collected  data.  The  primary  focus  of  the  study  was  the  impact  of  common 
instructional  interaction  strategies  on  the  construction  of  fragile  knowledge 
during  initial  procedural  skill  acquisition.  It  also  examined  whether  certain 
interaction  strategies  produced  particular  types  of  fragile  knowledge.  The 
secondary  focus  of  the  study  was  whether  explicit  instruction  on  executive 
control  strategies  can  influence  fragile  knowledge  construction. 
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Table  4.1 


Instructional  Totals  for 


Interaction 

Executive 

Controls 

Strateev 

Strategy 

Pretest 

Yes 

No 

M 

46.3 

34.2 

39.1 

None 

SD 

9.6 

13.0 

13.1 

(Control  Group) 

Posttest 

M 

59.7 

55.1 

60.0 

SD 

11.4 

11,4 

11.6 

n 

11 

16 

27 

Pretest 

M 

38.6 

46.2 

42.4 

Directive 

SD 

Posttest 

10.9 

10.2 

11.0 

M 

55.8 

55.0 

55.4 

SD 

10.6 

10.7 

10.5 

n 

13 

13 

26 

Pretest 

M 

44.8 

45.1 

44,9 

Questioning 

SD 

10.3 

11,4 

10.6 

Posttest 

M 

57.4 

55.7 

56.8 

SD 

11.4 

11.8 

11.5 

n 

31 

17 

48 

Pretest 

M 

46.0 

43.7 

45.1 

Mixed 

SD 

Posttest 

11.5 

16.1 

13.3 

M 

63.3 

65.2 

64.0 

SD 

11.5 

12.3 

11.7 

n 

29 

17 

46 

Yes 

No 

Pretest 

Executive 

M 

44.4 

42.2 

Controls 

SD 

10.8 

13.6 

Totals 

Posttest 

M 

59.5 

58.0 

SD 

11.5 

12.2 

n 

84 

63 
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Effect  of  Instructional  Interaction  Strategy:  Research  Question  #1 
Does  the  instructional  interaction  strategy  used  influence  the 
construction  of  fragile  knowledge  during  initial  procedural  skill  acquisition 
in  the  learner?  Yes,  a  main  effect  for  instructional  interaction  strategy  was 
found  in  the  ANCOVA  (F(3,138)=5.43,  p=.001)  see  Table  4.2  and  Figure  4.1 
below.  Post-hoc  Scheffe  comparison  indicated  that  the  mixed  strategy  group 
(the  group  that  received  the  combined  directive  and  questioning  interaction 
strategy)  scored  significantly  higher  on  the  posttest  than  the  other  groups. 

Table  4.2 

Analysis  of  Covariance  Summary  by  Independent  Variable 


DEP  VAR:  POSTTEST  COVARIATE:  PRETEST  N:  147 


SOURCE 

SUM  SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

INST  INTER  STRAT 

1590.18 

3 

530.06 

5.43 

.001* 

EXEC  CONTROLS 
.784 

INST  INTER  STRAT  by 

7.33 

1 

7.33 

.08 

EXEC  CONTROLS 

274.11 

3 

91.37 

.94  .425 

ERROR 

13466.57 

39 

97.58 

*  p  <  .05 

Note.  The  abbreviations  used  in  this  table  are  as  follows: 

INST  INTER  STRAT-Instructional  Interaction  Strategy.  This  is  one  of  four 
common  elemental  methods  or  tactics  for  conveying  instruction  to  the  learner 
EXEC  CONTROL S-Executive  Control  Strategies.  These  are  additional 
procedural  tools  used  to  control  and  cross-check  the  execution  of  the 
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procedure.  Effect  size  and  observed  power  at  the  .05  level  were  .264  and 
1.00  respectively. 

Figure  4.1 

Posttest  Scores  bv  Instructional  Interaction  Strategy 
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Types  of  Fragile  Knowledge  Construction:  Research  Question  la 
Do  certain  instructional  interaction  strategies  result  in  a  particular 
type  of  fragile  knowledge  construction  over  another  type?  No  significant 
differences  were  found  among  the  five  types  of  fragile  knowledge  for  any 
specific  instructional  interaction  strategy  (multivariate  F(12,423)  =  1.69, 
p=.067).  See  Figure  4.2  for  a  graphical  summary  of  the  results.  The 
multivariate  effect  size  and  power  at  the  .05  level  were  .046  and  .86 
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respectively.  No  further  analyses  were  conducted  on  types  of  fragile 
knowledge  construction. 

Figure  4.2 

Instances  of  Specific  Types  of  Fragile  Knowledge  by  Interaction  Strategy 
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Effect  of  Executive  Controls:  Research  Question  #2 
Does  the  explicit  teaching  of  executive  control  processes  (cross¬ 
checks  and  double-checks)  influence  the  construction  of  fragile  knowledge 
during  initial  procedural  skill  acquisition?  No,  there  was  no  main  effect  for 
executive  controls  (F(l,138)=.08,  p=.784).  See  Figure  4.3  for  a  graphical 
presentation  of  the  results  of  this  analysis.  Furthermore,  there  was  no 
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interaction  of  instructional  interaction  strategies  and  executive  controls 
(F(3,138)=94,  p=425). 

Figure  4.3 

Posttest  Scores  by  Executive  Controls 


Executive  Controls 


Now  that  we’ve  seen  the  numerical  results  of  the  statistical  analysis, 
what  does  all  this  mean  in  terms  of  the  study.  It’s  now  necessary  to  look  at 
the  results  and  interpret  them  in  light  of  the  instructional  theory  that  was 
presented  in  Chapters  1  and  2.  What  implications  do  these  results  have  for 
educational  professionals  who  are  concerned  about  fragile  knowledge 
construction  in  learners? 
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CHAPTER  5 


SUMMARY,  DISCUSSION,  AND  RECOMMENDATIONS 
Summary  of  the  Study 

This  study  focused  on  the  effects  of  four  common  instructional  interaction 
strategies  on  the  construction  of  fragile  knowledge.  It  also  examined  whether  the 
explicit  teaching  of  executiye  control  strategies  had  an  effect  on  fragile  knowledge 
construction.  This  chapter  proyides  a  summary  of  the  research  effort  to  answer  these 
questions,  a  discussion  of  the  results  with  implications  for  the  improyement  of 
instruction  inyolying  procedural  skill  acquisition,  and  recommendations  for  future 
studies  in  this  area. 

The  central  research  question  was  whether  the  strategy  used  to  interact  with 
the  learner  effects  the  stability  of  the  knowledge  the  learner  acquires?  If  so,  does  a 
particular  interaction  strategy  tend  to  create  a  specific  type  of  fragile  knowledge? 
Furthermore,  does  the  explicit  teaching  of  executiye  control  strategies  also  influence 
the  construction  of  fragile  knowledge  in  the  learner? 

The  treatment  consisted  of  common  yariations  in  the  way  instructors 
interacted  with  the  students.  Forty-three  instructors  were  trained  and  participated  in 
the  administration  of  the  treatment.  The  instructional  interactions  included  no  int 
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eraction  with  the  student,  directive  interactions,  questioning  interactions,  and  a  mix 
of  directive  and  questioning  interactions.  Explicit  instruction  on  executive  control 
strategies  was  also  provided  to  only  one  of  the  two  sub-groups  within  each 
instructional  interaction  strategy  treatment  group.  The  executive  control  strategies 
were  cross-checks  and  double-checks  of  the  computations  from  each  of  the 
navigational  procedures. 

Subjects  in  the  No  Instructional  Interaction  without  Executive  Controls  group 
were  informed  of  the  lesson  objective,  given  the  coursebook,  lesson  materials,  and 
sent  to  the  simulator.  Subjects  in  the  No  Instructional  Interaction  with  Executive 
Controls  groups  were  given  the  same  instructions  and  materials  except  that  their 
coursebook  provided  instruction  on  executive  controls. 

Subjects  in  the  Directive  without  Executive  Controls  group  were  explicitly 
told  how  to  accomplish  each  step  of  the  required  procedures.  Those  in  the  Directive 
with  Executive  Controls  group  received  the  same  style  of  instruction  with  the 
addition  of  explicit  teaching  of  specific  strategies  to  control  the  execution  of  the 
required  procedures. 

Participants  in  the  Questioning  without  Executive  Controls  group  were 
questioned  all  the  way  through  each  step  of  the  required  procedures.  The  questions 
focused  on  how  to  accomplish  each  step.  In  the  Questioning  with  Executive  Controls 
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group  subjects  were  also  questioned  through  strategies  to  control  the  execution  of  the 
required  procedures. 

The  Mixed  without  Executive  Controls  group  received  a  mixture  of  both  the 
directive  and  the  questioning  approaches.  The  Mixed  with  Executive  Controls  group 
received  the  same  approach  with  the  addition  of  instruction  on  how  to  control 
process  execution. 

An  ANCOVA  was  conducted  to  examine  the  data  for  main  effects  and  any 
interaction  between  the  independent  variables.  Pretest  results  were  used  as  a 
covariate  to  account  for  aptitude  effects.  Results  of  the  analysis  showed  that  the 
mixed  group  performed  significantly  better  than  the  other  three  groups  on  the 
posttest.  The  posttest  was  a  measure  of  the  amount  of  fragile  knowledge  a  subject 
possessed.  A  MANCOVA  for  types  of  fragile  knowledge  constructed  showed  no 
significant  differences  between  treatment  groups.  An  ANCOVA  for  the  effect  of 
executive  controls  on  fragile  knowledge  construction  showed  no  significant 
differences  between  treatment  groups. 

Discussion 

Are  certain  instructional  interaction  strategies  more  effective  than  others  in 
minimizing  fragile  knowledge  construction  in  procedural  skill  acquisition?  If  they 
are,  do  certain  interaction  strategies  have  a  tendency  to  produce  specific  types  of 
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fragile  knowledge?  Does  explicit  instruction  on  executive  control  strategies  also 
help  minimize  fragile  knowledge  construction? 

Epistemological  differences  have  fueled  the  debate  about  which  approaches 
to  instructional  interaction  are  the  most  effective.  Each  competing  educational 
philosophy  would  answer  these  questions  differently.  A  proponent  of  direct 
instruction  would  support  telling  learners  what  they  need  to  know.  A  proponent  of 
discovery  learning  would  want  to  put  them  in  an  environment  that  is  conducive  to 
learning  and  let  them  learn.  Proponents  of  the  Socratic  method  might  want  to 
challenge  learners  to  find  answers  by  using  questions  to  allow  them  to  probe  their 
understanding.  Adler  (1980)  and  Perkins  (1992)  suggest  a  mixed  approach.  They 
would  provide  didactic  instruction,  coach  them  by  observing  and  guiding,  and  follow 
up  with  the  Socratic  method.  With  all  these  apparently  competing  approaches  to 
achieving  the  same  objective,  who’s  right?  Which  method  is  most  likely  to  reduce 
fragile  knowledge  construction  during  procedural  skill  acquisition? 

The  results  of  this  study  support  the  importance  of  using  mixed  instructional 
interaction  strategies  to  minimize  the  construction  of  fragile  knowledge  in  learners. 
Subjects  in  this  study  that  were  initially  provided  a  directive  approach  to 
instructional  delivery  supported  by  a  shift  to  a  questioning  style  during  learner 
performance  performed  significantly  better  than  subjects  who  received  no 
instructional  interaction  or  only  the  directive  or  questioning  approach. 
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Furthermore,  the  group  that  received  no  instructional  interaction  performed 
equally  as  well,  if  not  better  in  some  cases,  than  the  directive  group  or  the 
questioning  group.  Another  way  of  looking  at  this  is  that  the  directive  or  questioning 
groups  performed  no  better  than  the  group  that  had  no  instructional  interaction  at  all. 
Based  on  post  treatment  interviews  one  possible  explanation  for  this  is  that  the 
directive  approach  did  not  accommodate  questions  from  the  students.  The  students 
may  have  had  questions,  but  the  instructors  addressed  them.  The  questioning 
approach  created  ambiguity  and  insecurity  because  this  interaction  strategy  created 
more  questions  in  the  learner  than  it  answered.  Subjects  frequently  stated  that  they 
understood  the  material  but  the  questioning  made  them  unsure  of  their 
understanding. 

Although  certain  groups  produced  more  instances  of  fragile  knowledge  than 
others,  no  treatment  group  produced  a  statistically  significant  amount  of  any  one 
specific  type  of  fragile  knowledge  than  any  other  group.  This  was  expected. 
Furthermore,  subjects  who  also  received  explicit  instruction  on  executive  control 
strategies  performed  better  than  other  subjects  in  most  instances;  however,  the 
difference  in  performance  was  not  statistically  significant. 

Limitations 

The  answer  to  the  first  question  of  whether  there  are  certain  interaction 
strategies  that  minimize  fragile  knowledge  construction  is  soundly  supported  by  the 
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results  of  this  study.  The  answers  to  the  other  two  questions  are  less  firmly  supported 
due  to  methodological  and  theoretical  problems  with  the  study.  Consequently,  these 
results  should  be  interpreted  with  caution. 

The  reason  for  caution,  in  regard  to  the  question  of  whether  certain 
interaction  strategies  produce  specific  types  of  fragile  knowledge,  is  that  the  analysis 
done  to  examine  this  question  was  accomplished  post-hoc.  The  primary  purpose  of 
the  instruments  used  in  the  research  was  to  detect  the  presence  of  fragile  knowledge 
not  to  discriminate  between  the  different  types.  Although  the  pre  and  posttests  were 
able  to  discriminate  between  the  different  types  of  fragile  knowledge,  they  did  not 
present  all  five  types  equal  opportunity  in  every  instance  throughout  the  entire 
measurement.  For  example,  some  questions  contained  the  correct  answer  and 
instances  of  only  one  type  of  fragile  knowledge  as  distracters.  The  effect  of  this 
would  be  to  provide  more  opportunity  to  find  significant  difference  with  this  type  of 
fragile  knowledge  simply  because  it  received  more  exposure  than  the  other  types. 

The  reason  for  caution  with  regard  to  the  second  main  research  question 
(whether  explicit  instruction  on  executive  control  strategies  has  an  effect  on  fragile 
knowledge  construction)  is  the  theoretical  interaction  between  executive  control 
strategies  themselves  and  the  resultant  effect  of  questioning  or  mixed  directive  and 
questioning  interaction  strategies.  If  subjects  simply  replicated  the  questions  posed 
by  the  instructor  during  the  initial  delivery,  then  they  in  effect  unintentionally 
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constructed  an  executive  control  strategy.  For  example,  if  an  instructor  questioned  a 
subject  on  where  to  get  a  certain  piece  of  data  to  solve  a  navigation  equation 
required  in  a  procedure,  and  the  subject  remembered  this  interaction,  then  the  subject 
could  use  this  to  repair  a  procedural  impasse  that  would  have  ordinarily  resulted  in 
inert  fragile  knowledge.  The  end  result  of  this  would  have  been  that  even  though 
subjects  were  not  given  explicit  instruction  on  executive  control  strategies,  they 
could  have  constructed  different,  but  effective,  executive  control  strategies  by 
themselves.  Consequently,  data  would  have  shown  no  significant  differences 
between  groups  that  did  or  did  not  receive  explicit  instruction  on  executive  control 
strategies.  The  results  of  this  study  support  this  interpretation. 

Finally,  although  this  study  is  theoretically  generalizable  to  similar  learning 
environments  that  involve  complex  procedural  skill  acquistion,  the  reader  must  be 
the  one  to  make  the  generalization.  This  study  examined  high  ability,  highly 
motivated  subjects  in  a  rigid  learning  environment;  other  environments  or  conditions 
may  vary  significantly. 

Recommendations 

Based  upon  the  results  of  this  study,  the  most  effective  way  to  minimize  the 
construction  of  fragile  knowledge  in  learners  during  initial  procedural  skill 
acquisition  is  to  open  the  delivery  with  a  directive  or  didactic  approach;  then,  shift  to 
a  questioning  strategy  as  the  learner  performs  the  procedure.  Other  strategies  may 
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work,  but  according  to  this  study,  they  will  have  a  tendency  to  produce  more  fragile 
knowledge. 

Initial  delivery  should  clearly  highlight  the  critical  attributes  of  the  procedure 
and  use  an  input-process-output  approach  to  teaching  each  step  in  the  procedure.  The 
instruction  should  explicitly  cover  where  to  get  input  data  from,  what  to  do  with  it, 
and  where  put  the  result.  This  then  becomes  the  input  for  the  next  step  of  the  process 
or  sub-process.  Once  this  has  been  accomplished,  instructional  interaction  should 
shift  to  a  questioning  style  while  the  learner  is  performing  the  procedure.  The 
questions  used  during  this  phase  should  center  on  where  one  gets  data  for  process 
input,  what  to  do  with  it,  what  to  do  with  the  results,  and  the  learner’s  evaluation  of 
the  performance  of  the  process. 

Instruction  involving  procedural  skill  acquisition  should  avoid  interaction 
strategies  with  the  learner  that  are  entirely  directive  or  declarative  in  nature  or 
interaction  that  is  entirely  question-based.  A  mix  of  the  two  strategies  in  the 
sequence  presented  above  will  produce  less  fragile  knowledge  in  the  learner. 

Future  Studies 

Future  studies  on  fragile  knowledge  need  to  address  the  issues  still 
unresolved  in  this  study  and  further  examine  why  mixed  strategies  are  better  than  the 
others.  Finding  answers  to  these  questions  would  clarify  our  understanding  of  fragile 
knowledge  and  help  to  minimize  its  consequences. 


68 


The  predisposition  of  certain  instructional  interaction  strategies  to  produce 
specific  types  of  fragile  knowledge  and  the  effect  of  executive  control  strategies  are 
issues  that  remain  unresolved  in  this  study.  Instrumentation  designed  specifically  to 
detect  the  types  of  fragile  knowledge  constructed  would  resolve  the  first  issue.  Also, 
studies  should  be  conducted  on  subjects  of  lower  ability  in  an  environment  that  is 
less  competitive.  Competitive  subjects  were  more  willing  and  better  able  to  produce 
conglomerative  fragile  knowledge  (combining  disparate  elements  of  knowledge  to 
be  able  to  produce  an  answer)  than  inert  fragile  knowledge  (simply  not  answering 
the  question).  Conducting  these  studies  would  also  help  establish  a  range  for  the 
magnitude  of  the  effect  of  interaction  strategies  on  fragile  knowledge  construction. 
With  regard  to  further  examination  of  the  effect  of  executive  control  strategies, 
changing  the  nature  of  the  questions  used  during  the  study  would  eliminate  the 
confounding  nature  of  the  interaction  between  the  questions  used  to  minimize  the 
construction  of  fragile  knowledge  and  the  executive  control  strategies  themselves. 
For  study  purposes,  instructors  should  not  use  questions  that  learners  can  construct 
control  strategies  from  by  mere  recall.  When  attempting  to  measure  the  effect  of 
executive  control  strategies,  avoiding  questions  like  “where  do  you  get  this 
information?”  or  “what  do  you  do  with  this  data?”  should  eliminate  the  confounding 
effects  in  the  study.  However,  it  may  be  necessary  to  answer  this  question  in  a 
completely  separate  study.  This  should  provide  a  more  definitive  answer  as  to 
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whether  the  explicit  instruction  of  executive  control  strategies  influences  the 
construction  of  fragile  knowledge. 

Conclusion 

Fragile  knowledge  can  have  significant  consequences.  Educational 
professionals  have  an  ethical  responsibility  to  minimize  or  eliminate  the  construction 
of  fragile  knowledge  in  learners.  This  study  has  shown  that  the  instructional 
interaction  strategies  we  use  with  learners  has  an  effect  on  the  amount  of  fragile 
knowledge  they  construct.  Therefore,  it  is  incumbent  upon  us  as  professional 
educators  to  prescribe,  and  properly  implement,  interaction  strategies  that  have  been 
shown  to  be  effective  against  fragile  knowledge  construction. 

Using  a  mixed  strategy  of  declarative  delivery  initially  and  question-based 
support  during  learner  performance  has  been  shown  to  be  an  effective  strategy. 
Avoidance  of  entirely  declarative  or  question-based  approaches  is  also  effective  in 
reducing  the  construction  of  fragile  knowledge.  While  questions  still  remain  as  to  the 
nature  of  fragile  knowledge  and  the  effect  of  executive  control  strategies,  the 
question  of  whether  or  not  how  instructors  or  instructional  systems  interact  with  the 
learner  effects  their  construction  of  fragile  knowledge  has  been  answered.  The  type 
or  style  of  interaction  with  a  learner  effects  the  stability  of  the  knowledge  they 
construct.  The  tragic  consequences  that  may  be  a  result  of  fragile  knowledge  can  be 
avoided  by  the  use  of  effective  instructional  interaction  strategies.  Can  we  afford  to 
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continue  to  use  strategies  that  are  economical  or  efficient  but  less  effective  in 
minimizing  fragile  knowledge  construction  and  its  consequences? 
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APPENDIX  A 


DATA  SUMMARY  FOR  ANCOVA 


Subject  # 

TreatGP 

InstStrat 

ExecCtrls 

Pretest 

Posttest 

1 

1 

None 

None 

10 

62 

2 

1 

None 

None 

33 

43 

3 

1 

None 

None 

33 

57 

4 

1 

None 

None 

29 

43 

5 

1 

None 

None 

29 

62 

6 

1 

None 

None 

48 

71 

7 

1 

None 

None 

33 

48 

8 

1 

None 

None 

38 

43 

9 

1 

None 

None 

43 

43 

10 

1 

None 

None 

33 

52 

11 

1 

None 

None 

14 

38 

12 

1 

None 

None 

62 

67 

13 

1 

None 

None 

52 

62 

14 

1 

None 

None 

24 

62 

15 

1 

None 

None 

29 

52 

16 

1 

None 

None 

38 

76 

17 

2 

None 

Yes 

43 

48 

18 

2 

None 

Yes 

57 

76 

19 

2 

None 

Yes 

57 

62 

20 

2 

None 

Yes 

52 

62 

21 

2 

None 

Yes 

33 

43 

22 

2 

None 

Yes 

43 

57 

23 

2 

None 

Yes 

38 

67 

24 

2 

None 

Yes 

48 

57 

25 

2 

None 

Yes 

38 

43 

26 

2 

None 

Yes 

62 

71 

27 

2 

None 

Yes 

38 

71 

28 

3 

Directive 

None 

29 

43 

29 

3 

Directive 

None 

62 

62 

30 

3 

Directive 

None 

52 

62 

31 

3 

Directive 

None 

48 

57 

72 


32 

3 

Directive 

None 

52 

62 

33 

3 

Directive 

None 

57 

67 

34 

3 

Directive 

None 

48 

57 

35 

3 

Directive 

None 

29 

29 

36 

3 

Directive 

None 

52 

62 

37 

3 

Directive 

None 

38 

52 

38 

3 

Directive 

None 

52 

62 

39 

3 

Directive 

None 

43 

57 

40 

3 

Directive 
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38 

43 

41 

4 

Directive 

Yes 

43 

67 

42 

4 

Directive 

Yes 

29 

38 

43 

4 

Directive 

Yes 

48 

48 

44 

4 

Directive 

Yes 

43 

43 

45 

4 

Directive 

Yes 

33 

38 

46 

4 

Directive 

Yes 

48 

62 

47 

4 

Directive 

Yes 

24 

62 

48 

4 

Directive 

Yes 

38 

57 

49 

4 

Directive 

Yes 

38 

57 

50 

4 

Directive 

Yes 

14 

57 

51 

4 

Directive 

Yes 

48 

62 

52 

4 

Directive 

Yes 

48 

67 

53 

4 

Directive 

Yes 

48 

67 

54 

5 

Questioning 

None 

43 

57 

55 

5 

Questioning 

None 

43 

48 

56 

5 

Questioning 

None 

48 

57 

57 

5 

Questioning 

None 

52 

71 

58 

5 

Questioning 

None 

52 

52 

59 

5 

Questioning 

None 

38 

43 

60 

5 

Questioning 

None 

52 

62 

61 

5 

Questioning 

None 

14 

24 

62 

5 

Questioning 

None 

48 

52 

63 

5 

Questioning 

None 

57 

62 

64 

5 

Questioning 

None 

38 

76 

65 

5 

Questioning 

None 

57 

67 

66 

5 

Questioning 

None 

48 

52 

67 

5 

Questioning 

None 

43 

48 

68 

5 

Questioning 

None 

43 

57 

69 

5 

Questioning 

None 

62 

62 

70 

5 

Questioning 

None 

29 

57 

71 

6 

Questioning 

Yes 

62 

67 

73 


72 

6 

Questioning 

Yes 

38 

48 

73 

6 

Questioning 

Yes 

38 

52 

74 

6 

Questioning 

Yes 

52 

62 

75 

6 

Questioning 

Yes 

52 

71 

16 

6 

Questioning 

Yes 

31 

57 

11 

6 

Questioning 

Yes 

57 

71 

78 

6 

Questioning 

Yes 

38 

38 

79 

6 

Questioning 

Yes 

52 

62 

80 

6 

Questioning 

Yes 

43 

48 

81 

6 

Questioning 

Yes 

57 

76 

82 

6 

Questioning 

Yes 

57 

67 

83 

6 

Questioning 

Yes 

33 

48 

84 

6 

Questioning 

Yes 

38 

48 

85 

6 

Questioning 

Yes 

52 

67 

86 

6 

Questioning 

Yes 

38 

43 

87 

6 

Questioning 

Yes 

62 

76 

88 

6 

Questioning 

Yes 

38 

38 

89 

6 

Questioning 

Yes 

29 

29 

90 

6 

Questioning 

Yes 

43 

48 

91 

6 

Questioning 

Yes 

57 

57 

92 

6 

Questioning 

Yes 

33 

62 

93 

6 

Questioning 

Yes 

38 

52 

94 

6 

Questioning 

Yes 

57 

62 

95 

6 

Questioning 

Yes 

43 

67 

96 

6 

Questioning 

Yes 

33 

57 

97 

6 

Questioning 

Yes 

33 

57 

98 

6 

Questioning 

Yes 

38 

67 

99 

6 

Questioning 

Yes 

57 

62 

100 

6 

Questioning 

Yes 

38 

62 

101 

6 

Questioning 

Yes 

52 

57 

102 

7 

Mixed 

None 

57 

71 

103 

7 

Mixed 

None 

38 

43 

104 

7 

Mixed 

None 

43 

62 

105 

7 

Mixed 

None 

52 

76 

106 

7 

Mixed 

None 

5 

67 

107 

7 

Mixed 

None 

43 

57 

108 

7 

Mixed 

None 

19 

57 

109 

7 

Mixed 

None 

38 

81 

110 

7 

Mixed 

None 

62 

81 

111 

7 

Mixed 

None 

43 

76 

74 
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7 

Mixed 

None 

62 

81 

113 

7 

Mixed 

None 

67 

76 

114 

7 

Mixed 

None 

62 

67 

115 

7 

Mixed 

None 

33 

62 

116 

7 

Mixed 

None 

38 

52 

117 

7 

Mixed 

None 

43 

52 

118 

7 

Mixed 

None 

38 

48 

119 

8 

Mixed 

Yes 

57 

76 

120 

8 

Mixed 

Yes 

43 

43 

121 

8 

Mixed 

Yes 

38 

57 

122 

8 

Mixed 

Yes 

52 

76 

123 

8 

Mixed 

Yes 

38 

71 

124 

8 

Mixed 

Yes 

67 

90 

125 

8 

Mixed 

Yes 

62 

67 

126 

8 

Mixed 

Yes 

43 

62 

127 

8 

Mixed 

Yes 

38 

67 

128 

8 

Mixed 

Yes 

43 

57 

129 

8 

Mixed 

Yes 

48 

48 

130 

8 

Mixed 

Yes 

52 

62 

131 

8 

Mixed 

Yes 

33 

67 

132 

8 

Mixed 

Yes 

62 

76 

133 

8 

Mixed 

Yes 

38 

57 

134 

8 

Mixed 

Yes 

43 

76 

135 

8 

Mixed 

Yes 

43 

71 

136 

8 

Mixed 

Yes 

33 

52 

137 

8 

Mixed 

Yes 

38 

62 

138 

8 

Mixed 

Yes 

24 

57 

139 

8 

Mixed 

Yes 

38 

38 

140 

8 

Mixed 

Yes 

76 

76 

141 

8 

Mixed 

Yes 

52 

52 

142 

8 

Mixed 

Yes 

33 

71 

143 

8 

Mixed 

Yes 

48 

52 

144 

8 

Mixed 

Yes 

48 

67 

145 

8 

Mixed 

Yes 

48 

57 

146 

8 

Mixed 

Yes 

38 

71 

147 

8 

Mixed 

Yes 

57 

57 
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APPENDIX  B 


DATA  SUMMARY  FOR  MANCOVA 


Subject  # 

Partial 

Conglom 

Misplaced 

Naive 

Inert 

1 

2 

2 

1 

2 

1 

2 

5 

2 

4 

1 

0 

3 

4 

2 

1 

2 

0 

4 

5 

2 

0 

1 

0 

5 

4 

5 

2 

1 

0 

6 

3 

2 

1 

0 

0 

7 

3 

2 

3 

3 

0 

8 

4 

4 

2 

2 

0 

9 

4 

6 

1 

0 

1 

10 

4 

2 

1 

2 

1 

11 

3 

5 

3 

2 

0 

12 

2 

4 

1 

0 

0 

13 

1 

5 

1 

1 

0 

14 

6 

1 

0 

1 

0 

15 

4 

2 

2 

1 

1 

16 

1 

3 

0 

1 

0 

17 

3 

4 

2 

1 

1 

18 

2 

3 

0 

0 

0 

19 

3 

2 

1 

2 

0 

20 

5 

0 

2 

1 

0 

21 

6 

5 

1 

1 

1 

22 

4 

1 

3 

1 

0 

23 

3 

2 

1 

1 

0 

24 

4 

2 

1 

2 

0 

25 

4 

4 

3 

1 

0 

26 

5 

0 

0 

1 

0 

27 

3 

2 

1 

1 

0 

28 

2 

5 

2 

2 

1 

29 

6 

1 

0 

1 

0 

30 

3 

2 

1 

1 

1 

76 


31  4  3 

32  4  2 

33  3  4 

34  2  4 

35  2  4 

36  3  2 

37  1  4 

38  4  1 

39  4  3 

40  2  6 

41  1  2 

42  6  3 

43  2  3 

44  3  4 

45  4  2 

46  2  1 

47  5  2 

48  3  2 

49  2  1 

50  5  3 

51  4  1 

52  4  0 

53  3  2 

54  4  3 

55  6  0 

56  2  2 

57  3  1 

58  3  2 

59  2  6 

60  3  2 

61  5  0 

62  6  1 

63  4  2 

64  1  1 

65  3  1 

66  2  6 

67  4  3 


1 

1 

0 

2 

1 

2 

2 

1 

1 

3 

1 

1 

1 

2 

2 

3 

0 

2 

2 

0 

1 

3 

1 

1 

3 

2 

1 

1 

2 

1 

0 

2 

1 

1 

2 

1 

2 


1 

1 

0 

1 

3 

1 

1 

2 

1 

1 

3 

3 

3 

2 

2 

2 

1 

2 

2 

1 

2 

0 

0 

1 

2 

2 

0 

0 

1 

2 

0 

1 

1 

2 

1 

1 

2 


0 

0 

1 

0 

5 

0 

2 

0 

0 

0 

0 

0 

0 

1 

4 

0 

0 

0 

2 

0 

0 

0 

1 

0 

0 

1 

1 

4 

1 

0 

11 

0 

0 

0 

0 

0 

0 
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68  4  0 

69  2  4 

70  5  1 

71  3  2 

72  6  1 

73  4  1 

74  4  2 

75  3  0 

76  4  2 

77  3  1 

78  6  1 

79  5  0 

80  3  3 

81  2  1 

82  1  2 

83  6  1 

84  4  3 

85  2  3 

86  5  3 

87  4  0 

88  2  3 

89  9  2 

90  5  4 

91  3  4 

92  3  3 

93  3  2 

94  3  3 

95  4  2 

96  5  2 

97  3  2 

98  2  4 

99  5  2 

100  6  0 

101  4  2 

102  2  2 

103  5  2 

104  3  3 


3 

1 

2 

2 

1 

3 

0 

1 

1 

1 

3 

1 

2 

1 

2 

2 

1 

1 

2 

0 

2 

1 

2 

2 

2 

2 

1 

1 

2 

2 

1 

0 

1 

3 
2 

4 
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2 

1 

1 

0 

2 

2 

2 

2 

2 

1 

3 

2 

3 

1 

2 

1 

3 

0 
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1 
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2 
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2 
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0 
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1 

1 

0 

0 

2 
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0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 


0 

1 

0 


0 

5 

0 

0 

0 

0 

1 

0 

0 

0 

2 

0 

0 

0 

0 

0 

0 

0 
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106 

107 

108 
109 

no 

111 

112 

113 

114 

115 

116 

117 
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119 

120 
121 
122 

123 
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125 
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127 

128 

129 

130 

131 
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133 

134 

135 

136 

137 

138 

139 

140 

141 


2 

3 

3 

5 

2 

3 

2 

2 

2 

1 

2 

3 

3 

5 

2 

3 

2 

3 

4 
2 

4 
1 
1 
3 

5 
3 
5 

3 

4 
3 

3 
2 

5 

4 

6 
3 

5 


0 

1 

1 

0 

0 

1 

0 

0 

1 

3 
2 

4 

4 

3 
2 

5 

4 
1 
1 
0 
1 

5 

4 
2 
3 
1 
0 
0 
2 
1 

3 

5 
1 
2 

4 
0 
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2 

1 

3 

2 

1 

0 

1 

0 

1 

1 

2 

1 

2 

0 

0 

2 

2 

0 

1 

0 

1 

1 

1 

2 

1 

2 

1 

2 

2 

0 

0 

1 

1 

2 

0 

0 

1 


0 

2 
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3 

2 
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0 

0 

0 

0 

0 

0 

1 

1 

2 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 
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0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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142  1  3  1  1  0 

143  5  4  0  1  0 

144  2  3  1  1  0 

145  2  3  2  2  0 

146  1  4  0  0  1 

147  3  3  2  1  0 
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APPENDIX  C 


Pre-Study  Questionnaire 

Note 

This  data  will  remain  strictly  confidential.  Under  no  circumstances  will  this  be  divulged  to 
anyone  other  than  you  without  your  knowledge  and  prior  written  consent. 

Name: 

Age: 

Gender:  M  F 

Nationality:  U,  S.  Other _ 

SAT  Composite  Score: 

High  School  GPA: 

Is  English  your  native  language?  Yes  No 
Previous  Pilot  Flight  Experience  (ex:  Private  Pilot,  40  hours) 

Previous  Navigation  Ex^perience  (ex:  Navy  Sea  Cadets,  2  yrs;  NJROTC,  Orienteering,  etc.): 


Are  you  Pilot  or  Nav  Qualified?  Yes  No  Don’t  Know 

If  you  couldn’t  fly  for  the  Air  Force  would  you  leave  the  Academy?  Yes  No 

Were  your  favorite  courses  in  High  School  Math  and  Science  type  courses?  Yes 

Do  you  intend  to  pursue  a  technical/engineering  major  at  the  Academy?  Yes 

Did  you  attend  a  Preparatory  School?  Yes  No 

Are  you  Prior  S  ervice?  Yes  No 

If  yes,  were  you  an  Air  Crewmember?  Yes  No 

or  an  Instructor/Trainer  of  Air  Crewmembers?  Y es 

Procedural  Knowledge  Memory  Test  Score _ 


No 

No 


No 
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i^PPENDIX  D 

Name: _ 

Aeronautical  Knowledge  Test  #1 
Choose  the  best  answer.  More  than  one  answer  per  question  may  be  correct. 

1 .  Given  the  information  shown  in  Figures  1  8c  2.,  what  heading  should  the  pilot  fly  at  the 
Departure  Point? 

A.  254  degrees 

B.  256  degrees 

C.  266  degrees 

D.  268  degrees 

2.  Given  the  indications  shown  in  Figure  2.,  which  of  the  following  most  accurately  depicts  the 
aircraft’s  position?  (Show  your  work  on  Figure  1.) 


3.  Based  on  the  results  of  the  fix  resolved  and  plotted  on  Figure  1 .  and  the  indications  in  Figure 
2.,  which  magnetic  heading  should  be  flown  to  correct  to  course? 

A.  265  degrees 

B.  263  degrees 

C.  254  degrees 

D.  252  degrees 

4.  Using  the  information  provided  in  Figure  3.,  which  of  the  following  is  the  most  accurate  ETA 
for  Point  B? 

A.  1216Z 

B.  1217Z 

C.  1234Z 

D.  1235Z 
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5.  Using  the  indications  shown  in  Figure  3.,  which  of  the  following  depictions  most  accurately 
represents  where  the  aircraft  will  be  at  1230Z? 


6.  Given  the  indications  shown  in  Figure  4.,  which  of  the  following  most  accurately  depicts  the 
aircraft’s  position  at  1230Z? 


7.  The  distance  between  each  tick  mark  on  a  line  of  longitude  on  a  JNC  chart 

A.  equals  2  NM 

B.  equals  5  NM 

C.  equals  60  NM 

D.  cannot  be  determined  because  they  are  not  equidistant  due  to  convergence 

8.  Variation  is  the  difference  betw’een 

A.  True  North  and  True  Heading 

B.  True  North  and  Magnetic  North 

C.  True  Course  and  Magnetic  Course 

D.  subsequent  readings  of  the  same  instrument 
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9.  The  difference  between  Desired  Course  and  Actual  Course  is  called 

A.  Variation 

B.  Deviation 

C.  Drift 

D.  Slant  Error 

10.  What  effect  will  a  25  knot  headwind  instead  of  a  50  knot  headwind  have  on  your  Proposed 
ETA  to  your  final  destination? 

A.  It  will  take  only  half  as  much  time  to  get  there 

B.  It  will  take  twice  as  much  time  to  get  there 

C.  Arrive  later  than  originally  planned 

D.  Arrive  earlier  than  originally  planned 

11.  To  maintain  a  planned/desired  ground  track  on  a  JNC  chart,  a  pilot  must  correct  for 

A.  crosstrack  error  and  precession 

B.  drift  and  precession 

C.  variation  and  precession 

D.  variation  and  drift 

12.  If  a  pilot  must  fly  a  heading  of  260  degrees  to  maintain  a  track  of  270  degrees  and  now  is  10 
minutes  earlier  to  the  arrival  gate  than  originally  planned  which  general  direction  is  the  wind 
coming  from? 

A.  Northwest 

B.  Northeast 

C.  Southwest 

D.  Southeast 

13.  A  Fix  is  a(n) 

A.  estimate  of  the  aircraft’s  current  position 

B.  measurement  of  the  aircraft’s  current  position 

C.  estimate  of  the  aircraft’s  future  position 

D.  measurement  of  the  aircraft’s  future  position 

14.  A  Dead  Reckoning  (DR)  position  is  a(n) 

A.  estimate  of  the  aircraft’s  current  position 

B.  measurement  of  the  aircraft’s  current  position 

C.  estimate  of  the  aircraft’s  future  position 

D.  measurement  of  the  aircraft’s  future  position 

1 5.  When  taking  a  TACAN  fix,  where  do  we  get  the  variation  from? 

A.  the  Navigation  Computer  System  (NCS) 

B.  the  line  of  variation  halfway  between  the  2  points  on  the  chart 

C.  the  line  of  variation  closest  to  the  TACAN  station  on  the  chart 

D.  the  IFR  Supplement 
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16.  What  is  the  distance  between  N  21-18  W  105-15  and  N  22-18  W  105-15? 

A.  60  NM 

B.  6NM 

C.  INM 

D.  .6NM 

17.  How  far  will  the  aircraft  have  traveled  if  it’s  been  maintaining  400  GS  for  9  minutes 

A.  59.9  NM 

B.  approximately  63  NM 

C.  a  distance  equal  to  about  one  degree  of  latitude 

D.  about  66  NM 

18.  If  your  Desired  Course  is  180  degrees  and  you  experience  1  degree  more  right  drift  than  you 
computed  in  your  DR,  where  would  you  expect  your  fix  to  fall  in  relation  to  your  DR  assuming 
240  NM  between  your  last  fix  and  current  DR? 

A.  1  NM  due  east  of  the  DR 

B. .  4  NM  due  east  of  the  DR 

C.  1  NM  due  west  of  the  DR 

D.  4  NM  due  west  of  the  DR 

19.  The  angular  difference  measured  in  degrees  between  two  points  on  a  JNC  chart  is  called 

A.  True  Course 

B.  True  Heading 

C.  Magnetic  Course 

D.  Magnetic  Heading 

20.  For  fixing  purposes,  the  RADAR  is  oriented  to 

A.  True  Course 

B.  True  Heading 

C.  True  North 

D.  Magnetic  Heading 

21 .  Which  of  the  following  navigational  facilities  does  not  provide  enough  information  to 
establish  a  fix? 

A.  VOR 

B.  VORTAC 

C.  VOR/DME 

D.  TAC 
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APPENDIX  E 


Name: _ _ _ 

Aeronautical  Knowledge  Test  #2 

Choose  the  best  answer.  More  than  one  answer  per  question  may  be  correct, 

1 .  Given  the  information  shown  in  Figures  1 .  &  2.,  what  heading  should  the  pilot  fly  at  the 
Departure  Point? 

A.  268  degrees 

B.  266  degrees 

C.  256  degrees 

D.  254  degrees 


2.  Given  the  indications  shown  in  Figure  2.,  w^hich  of  the  following  most  accurately  depicts  the 
aircraft’s  position?  (Show  your  work  on  Figure  1 .) 


3.  Based  on  the  results  of  the  fi.x  resolved  and  plotted  on  Figure  1.  and  the  indications  in  Figure 
2.,  which  magnetic  heading  should  be  flown  to  correct  to  course? 

A.  252  degrees 

B.  254  degrees 

C.  263  degrees 

D.  265  degrees 

4.  Using  the  information  provided  in  Figure  3.,  which  of  the  following  is  the  most  accurate  ETA 
for  Point  B? 

A.  1235Z 

B.  1234Z 

C.  1217Z 

D.  1216Z 
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5.  Using  the  indications  shown  in  Figure  3.,  which  of  the  following  depictions  most  accurately 
represents  where  the  aircraft  will  be  at  1230Z? 


.  j  V 


6.  Given  the  indications  shown  in  Figure  4.,  which  of  the  following  most  accurately  depicts  the 
aircraft’s  position  at  1230Z? 


7.  Which  of  the  following  navigational  facilities  does  not  provide  enough  information  to 
establish  a  fix? 

A.  VOR 

B.  VORTAC 

C.  VOR/DME 

D.  TAG 

8.  For  fixing  purposes,  the  RADAR  is  oriented  to 

A.  True  Course 

B.  True  Heading 

C.  True  North 

D.  Magnetic  Heading 
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9.  The  angular  difference  measured  in  degrees  between  Uvo  points  on  a  JNC  chart  is  called 

A.  True  Course 

B.  True  Heading 

C.  Magnetic  Course 

D.  Magnetic  Heading 

10.  If  your  Desired  Course  is  1 80  degrees  and  you  e.Kperience  1  degree  more  right  drift  than  you 
computed  in  your  DR,  where  would  you  expect  your  fix  to  fall  in  relation  to  your  DR  assuming 
240  NM  between  your  last  fix  and  current  DR? 

A.  1  NM  due  east  of  the  DR 

B.  4  NM  due  east  of  the  DR 

C.  1  NM  due  west  of  the  DR 

D.  4  NM  due  west  of  the  DR 

1 1 .  How  far  will  the  aircraft  have  traveled  if  it’s  been  maintaining  400  GS  for  9  minutes? 

A.  59,9  NM 

B.  approximately  63  NM 

C.  a  distance  equal  to  about  one  degree  of  latitude 

D.  about  66  NM 

12.  What  is  the  distance  between  N  21-18  W  105-15  and  N  22-18  W  105-15? 

A.  60  NM 

B.  6NM 

C.  INM 

D.  .6NM 

13.  When  taking  a  TACAN  fix,  where  do  we  get  the  variation  from? 

A.  the  Navigation  Computer  System  (NCS) 

B.  the  line  of  variation  halfway  between  the  2  points  on  the  chart 

C.  the  line  of  variation  closest  to  the  TACAN  station  on  the  chart 

D.  the  IFR  Supplement 

14.  A  Dead  Reckoning  (DR)  position  is  a(n) 

A.  estimate  of  the  aircraft’s  current  position 

B.  measurement  of  the  aircraft’s  current  position 

C.  estimate  of  the  aircraft’s  future  position 

D.  measurement  of  the  aircraft’s  future  position 

15.  A  Fix  is  a(n) 

A.  estimate  of  the  aircraft’s  current  position 

B.  measurement  of  the  aircraft’s  current  position 

C.  estimate  of  the  aircraft’s  future  position 

D.  measurement  of  the  aircraft’s  future  position 
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16.  If  a  pilot  must  fly  a  heading  of  260  degrees  to  maintain  a  track  of  270  degrees  and  now  is  10 
minutes  earlier  to  the  arrival  gate  than  originally  planned  which  general  direction  is  the  wind 
coming  from? 

A.  Northwest 

B.  Northeast 

C.  Southwest 

D.  Southeast 

17.  To  maintain  a  planned/desired  ground  track  on  a  JNC  chart,  a  pilot  must  correct  for 

A.  crosstrack  error  and  precession 

B.  drift  and  precession 

C.  variation  and  precession 

D.  variation  and  drift 

i  8.  What  effect  will  a  25  knot  headwind  instead  of  a  50  knot  headwind  have  on  your  Proposed 
ETA  to  your  final  destination? 

A.  It  will  take  only  half  as  much  time  to  get  there 

B.  It  will  take  twice  as  much  time  to  get  there 

C.  You  will  arrive  later  than  originally  planned 

D.  You  will  arrive  earlier  than  originally  planned 

19.  The  difference  between  Desired  Course  and  Actual  Course  is  called 

A.  Variation 

B.  Deviation 

C.  Drift 

D.  Slant  Error 

20.  Variation  is  the  difference  between 

A.  True  North  and  True  Heading 

B.  True  North  and  Magnetic  North 

C.  True  Course  and  Magnetic  Course 

D.  subsequent  readings  of  the  same  instrument 

21.  The  distance  between  each  tick  mark  on  a  line  of  longitude  on  a  JNC  chart 

A.  equals  2  NM 

B.  equals  5  NM 

C.  equals  60  NM 

D.  cannot  be  determined  because  they  are  not  equidistant  due  to  convergence 
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APPENDIX  F 


STEM  AND  LEAF  PLOTS  OF  STUDY  DATA 


POSTTEST 


Valid  cases;  147.0  Missing  cases:  .0  Percent  missing;  .0 


Mean  58.8231  StdErr  .9736  Min 
Median  62.0000  Variance  139.3521  Max 
5%  Trim  59.0484  Std  Dev  11.8047  Range 
95%  Cl  for  Mean  (56.8989,  60.7474)  IQR 


24.0000  Skewness  -.2573 
90.0000  SESkew  .2000 
66.0000  Kurtosis  -.0064 
15.0000  SEKurt  .3975 


Frequency  Stem&  Leaf 

3.00  Extremes  (24),  (29) 

6.00  3  .  888888 

13.00  4  *  3333333333333 

12.00  4.  888888888888 

13.00  5  *  2222222222222 

26.00  5 .  77777777777777777777777777 

28.00  6  *  2222222222222222222222222222 

18.00  6.  777777777777777777 

11.00  7  *  11111111111 

13.00  7.  6666666666666 

3.00  8  *  111 

1.00  Extremes  (90) 


Stem  width:  10 
Each  leaf;  1  case(s) 

POSTTEST 

Statistic  df  Significance 
K-S  (Lilliefors)  .0810  147  .0196 
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STEM  AND  LEAF  PLOTS  FOR  MANGO VA  (DEPENDENT  VARIABLES) 
CONGLOM 

Valid  cases:  147.0  Missing  cases;  .0  Percent  missing:  .0 


Mean  2.2653  StdErr  .1249  Min  .0000  Skewness  .4160 
Median  2.0000  Variance  2.2921  Max  6.0000  S  E  Skew  .2000 

5%  Trim  2.2090  StdDev  1.5140  Range  6.0000  Kurtosis  -.3645 

95%  Cl  for  Mean  (2.0185,  2.5121)  IQR  2.0000  SEKurt  .3975 


Frequency  Stem  &  Leaf 

19.00  0  *  0000000000000000000 

.00  0 . 

28.00  1  *  0000000000000000000000000000 

.00  1  . 

43.00  2  *  0000000000000000000000000000000000000000000 

.00  2 . 

25.00  3  *  0000000000000000000000000 

.00  3  . 

20.00  4*  00000000000000000000 

.00  4  . 

8.00  5  *  00000000 

4.00  Extremes  (6.0) 

Stem  width:  1 
Each  leaf:  1  case(s) 

INERT 

Valid  cases:  147.0  Missing  cases:  .0  Percent  missing;  .0 


Mean  .4218  StdErr  .1012  Min  .0000  Skewness  5.5721 
Median  .0000  Variance  1.5058  Max  11.0000  S  E  Skew  .2000 
5%  Trim  .2139  StdDev  1.2271  Range  11.0000  Kurtosis  40.2612 

95%  Cl  for  Mean  (.2217,  .6218)  IQR  .0000  S  E  Kurt  .3975 
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Frequency  Stem  &  Leaf 


114.00  0*  00000000000000000000000000000000000000 

33.00  Extremes  (1),  (2),  (4),  (5),  (1 1) 

Stem  width;  10 
Each  leaf:  3  case(s) 

MISPLACE 

Valid  cases:  147.0  Missing  cases:  .0  Percent  missing:  .0 


Mean  1.3469  StdErr  .0758  Min 

Median  1.0000  Variance  .8446  Max 

5%  Trim  1.3148  Std  Dev  ,9190  Range 
95%  Cl  for  Mean  (1.1971,  1.4967)  IQR 


.0000  Skewness  .3804 
4.0000  S  E  Skew  .2000 
4.0000  Kurtosis  -.1530 
1.0000  S  EKurt  .3975 


Frequency  Stem  &  Leaf 


26.00 

0  * 

0000000000000 

.00 

Ot 

.00 

Of 

.00 

Os 

.00 

0  . 

61.00 

1  * 

000000000000000000000000000000 

.00 

1  t 

.00 

1  f 

.00 

Is 

.00 

1  . 

45.00 

2  * 

0000000000000000000000 

.00 

2t 

.00 

2f 

.00 

2s 

.00 

2  . 

13.00 

3  * 

000000 

2.00  Extremes  (4.0) 
Stem  width;  1 
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Each  leaf; 


2  case(s) 


NAIVE 


Valid  cases: 


147.0  Missing  cases:  .0  Percent  missing:  .0 


Mean  1.2585  StdErr  .0710  Min 
Median  1.0000  Variance  .7409  Max 
5%  Trim  1.2317  Std  Dev  .8608  Range 
95%  Cl  for  Mean  (1.1 182,  1.3988)  IQR 


.0000  Skewness  .1913 
3.0000  SESkew  .2000 
3.0000  Kurtosis  -.6212 
1.0000  SEKurt  .3975 


Frequency  Stem  &  Leaf 


29.00 

0  * 

00000000000000 

.00 

Ot 

.00 

Of 

.00 

0  s 

.00 

0  . 

62.00 

1  * 

0000000000000000000000000000000 

.00 

1  t 

.00 

1  f 

.00 

1  s 

.00 

1  . 

45.00 

2  * 

0000000000000000000000 

.00 

2t 

.00 

2f 

.00 

2s 

.00 

2. 

11.00 

3  * 

00000 

Stem  width:  1 
Each  leaf:  2  case(s) 
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L 


PARTIAL 


Valid  cases;  147.0  Missing  cases:  .0  Percent  missing:  .0 


Mean  3.4014  StdErr  .1185  Min 
Median  3.0000  Variance  2.0638  Max 
5%  Trim  3.3677  StdDev  1.4366  Range 
95%  Cl  for  Mean  (3.1672,  3.6355)  IQR 


1.0000  Skewness  .5168 
9.0000  S  E  Skew  .2000 
8.0000  Kurtosis  .4754 
2.0000  SEKurt  .3975 


Frequency  Stem  &  Leaf 

11.00  1  *  00000000000 

.00  1  . 

31.00  2  *  0000000000000000000000000000000 

.00  2 . 

41.00  3  *  00000000000000000000000000000000000000000 

.00  3  . 

31.00  4  *  0000000000000000000000000000000 

.00  4 . 

21.00  5  *  000000000000000000000 

.00  5  . 

11.00  6  *  00000000000 

1.00  Extremes  (9.0) 

Stem  width:  1 
Each  leaf:  1  case(s) 
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APPENDIX  G 

Consent  Form 

Note 

This  form  is  a  privileged  record  and  will  be  maintained 
within  professional  standards  of  confidentiality 

or  penalty  of  any  kind.  discontinue  participation  at  any  time  without  prejudice 

Purpose:  To  assess  whether  or  not  how  a  person  is  taught  effects  what  they  learn 

PMi“p“„7r  be':,!':  “  '■r «““y 

the  participant’s  level  of  aeronautical  knowIeHof-  *  provide  a  baseline  on 

i=Sfsa>d:““55r“"^ 

be  administered  after  the  flight  to  collect  data  on  the^partklp'^s 

smimsm- 

173364,  Denver,  CO  80217-3364,  telephone  (303)  556-2770.  ’ 


Signature 


Date 
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